{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02134a5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "!pip install -Uqq nixtla"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6d8f223",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "from nixtla.utils import in_colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c6c0333",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "IN_COLAB = in_colab()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce98fab5",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    from nixtla.utils import colab_badge\n",
    "    from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da753996-54f8-4244-a34e-7316b0c01827",
   "metadata": {},
   "source": [
    "# Fine-tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75a62889-d81e-462e-b235-c1eba1096da9",
   "metadata": {},
   "source": [
    "Fine-tuning is a powerful process for utilizing TimeGPT more effectively. Foundation models such as TimeGPT are pre-trained on vast amounts of data, capturing wide-ranging features and patterns. These models can then be specialized for specific contexts or domains. With fine-tuning, the model's parameters are refined to forecast a new task, allowing it to tailor its vast pre-existing knowledge towards the requirements of the new data. Fine-tuning thus serves as a crucial bridge, linking TimeGPT's broad capabilities to your tasks specificities.\n",
    "\n",
    "Concretely, the process of fine-tuning consists of performing a certain number of training iterations on your input data minimizing the forecasting error. The forecasts will then be produced with the updated model. To control the number of iterations, use the `finetune_steps` argument of the `forecast` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "448eaf77-0a40-4b5b-88a2-31de99f404bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/tutorials/06_finetuning.ipynb)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#| echo: false\n",
    "if not IN_COLAB:\n",
    "    load_dotenv()    \n",
    "    colab_badge('docs/tutorials/06_finetuning')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10ec4f03",
   "metadata": {},
   "source": [
    "## 1. Import packages\n",
    "First, we import the required packages and initialize the Nixtla client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98942108-d427-42d6-81f8-fa0bb5859395",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from nixtla import NixtlaClient\n",
    "from utilsforecast.losses import mae, mse\n",
    "from utilsforecast.evaluation import evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64178d1c-957e-4a04-ab64-fde332b1840c",
   "metadata": {},
   "outputs": [],
   "source": [
    "nixtla_client = NixtlaClient(\n",
    "    # defaults to os.environ.get(\"NIXTLA_API_KEY\")\n",
    "    api_key = 'my_api_key_provided_by_nixtla'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b57a38e6",
   "metadata": {},
   "source": [
    "> 👍 Use an Azure AI endpoint\n",
    "> \n",
    "> To use an Azure AI endpoint, remember to set also the `base_url` argument:\n",
    "> \n",
    "> `nixtla_client = NixtlaClient(base_url=\"you azure ai endpoint\", api_key=\"your api_key\")`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5cd61549-0b00-4a42-a98e-239fa4fae5e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    nixtla_client = NixtlaClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c2e5387",
   "metadata": {},
   "source": [
    "## 2. Load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b78cc83e-7d34-4c37-906d-8c7ed1a977fb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1949-01-01</td>\n",
       "      <td>112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1949-02-01</td>\n",
       "      <td>118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1949-03-01</td>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1949-04-01</td>\n",
       "      <td>129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1949-05-01</td>\n",
       "      <td>121</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    timestamp  value\n",
       "0  1949-01-01    112\n",
       "1  1949-02-01    118\n",
       "2  1949-03-01    132\n",
       "3  1949-04-01    129\n",
       "4  1949-05-01    121"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/air_passengers.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09be4766",
   "metadata": {},
   "source": [
    "## 3. Fine-tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f5b9060",
   "metadata": {},
   "source": [
    "Here, `finetune_steps=10` means the model will go through 10 iterations of training on your time series data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a683abc7-190c-40a6-a4e8-41a4c64bd773",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "INFO:nixtla.nixtla_client:Querying model metadata...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "timegpt_fcst_finetune_df = nixtla_client.forecast(\n",
    "    df=df, h=12, finetune_steps=10,\n",
    "    time_col='timestamp', target_col='value',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac469746",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "545ffdac-f166-417b-993f-78f51b0db6a1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 73796 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df, timegpt_fcst_finetune_df, \n",
    "    time_col='timestamp', target_col='value',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62fc9cba-7c6e-4aef-9c68-e05d4fe8f7ba",
   "metadata": {},
   "source": [
    "Keep in mind that fine-tuning can be a bit of trial and error. You might need to adjust the number of `finetune_steps` based on your specific needs and the complexity of your data. Usually, a larger value of `finetune_steps` works better for large datasets.\n",
    "\n",
    "It's recommended to monitor the model's performance during fine-tuning and adjust as needed. Be aware that more `finetune_steps` may lead to longer training times and could potentially lead to overfitting if not managed properly. \n",
    "\n",
    "Remember, fine-tuning is a powerful feature, but it should be used thoughtfully and carefully."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c546351",
   "metadata": {},
   "source": [
    "For a detailed guide on using a specific loss function for fine-tuning, check out the [Fine-tuning with a specific loss function](https://docs.nixtla.io/docs/tutorials-fine_tuning_with_a_specific_loss_function) tutorial.\n",
    "\n",
    "Read also our detailed tutorial on [controlling the level of fine-tuning](https://docs.nixtla.io/docs/tutorials-finetune_depth_finetuning) using `finetune_depth`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
