{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "!pip install -Uqq nixtla"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "from nixtla.utils import in_colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "IN_COLAB = in_colab()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    from nixtla.utils import colab_badge\n",
    "    from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Improve Forecast Accuracy with TimeGPT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we demonstrate how to use TimeGPT for forecasting and explore three common strategies to enhance forecast accuracy. We use the hourly electricity price data from Germany as our example dataset. Before running the notebook, please initiate a NixtlaClient object with your api_key in the code snippet below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Result Summary\n",
    "\n",
    "| Steps | Description                  | MAE  | MAE Improvement (%) | RMSE  | RMSE Improvement (%) |\n",
    "|-------|------------------------------|------|---------------------|-------|----------------------|\n",
    "| 0     | Zero-Shot TimeGPT            | 18.5 | N/A                 | 20.0  | N/A                  |\n",
    "| 1     | Add Fine-Tuning Steps        | 11.5 | 38%                 | 12.6  | 37%                  |\n",
    "| 2     | Adjust Fine-Tuning Loss      | 9.6  | 48%                 | 11.0  | 45%                  |\n",
    "| 3     | Fine-tune more parameters    | 9.0  | 51%                 | 11.3  | 44%                  |\n",
    "| 4     | Add Exogenous Variables      | 4.6  | 75%                 | 6.4   | 68%                  |\n",
    "| 5     | Switch to Long-Horizon Model | 6.4  | 65%                 | 7.7   | 62%                  |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/tutorials/22_how_to_improve_forecast_accuracy.ipynb)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#| echo: false\n",
    "if not IN_COLAB:\n",
    "    load_dotenv()\n",
    "    colab_badge('docs/tutorials/22_how_to_improve_forecast_accuracy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we install and import the required packages, initialize the Nixtla client and create a function for calculating evaluation metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from utilsforecast.evaluation import evaluate\n",
    "from utilsforecast.plotting import plot_series\n",
    "from utilsforecast.losses import mae, rmse\n",
    "from nixtla import NixtlaClient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nixtla_client = NixtlaClient(\n",
    "    # api_key = 'my_api_key_provided_by_nixtla'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. load in dataset\n",
    "In this notebook, we use hourly electricity prices as our example dataset, which consists of 5 time series, each with approximately 1700 data points. For demonstration purposes, we focus on the German electricity price series. The time series is split, with the last 48 steps (2 days) set aside as the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short-with-ex-vars.csv')\n",
    "df['ds'] = pd.to_datetime(df['ds'])\n",
    "df_sub = df.query('unique_id == \"DE\"')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1632, 12), (48, 12))"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train = df_sub.query('ds < \"2017-12-29\"')\n",
    "df_test = df_sub.query('ds >= \"2017-12-29\"')\n",
    "df_train.shape, df_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 71400 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_series(df_train[['unique_id','ds','y']][-200:], forecasts_df= df_test[['unique_id','ds','y']].rename(columns={'y': 'test'}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Benchmark Forecasting using TimeGPT\n",
    "We used TimeGPT to generate a zero-shot forecast for the time series. As illustrated in the plot, TimeGPT captures the overall trend reasonably well, but it falls short in modeling the short-term fluctuations and cyclical patterns present in the actual data. During the test period, the model achieved a Mean Absolute Error (MAE) of 18.5 and a Root Mean Square Error (RMSE) of 20. This forecast serves as a baseline for further comparison and optimization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Querying model metadata...\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Restricting input...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "fcst_timegpt = nixtla_client.forecast(df = df_train[['unique_id','ds','y']],\n",
    "                                      h=2*24,\n",
    "                                      target_col = 'y',\n",
    "                                      level = [90, 95])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metrics = [mae, rmse]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>metric</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DE</td>\n",
       "      <td>mae</td>\n",
       "      <td>18.519004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DE</td>\n",
       "      <td>rmse</td>\n",
       "      <td>20.037751</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id metric    TimeGPT\n",
       "0        DE    mae  18.519004\n",
       "1        DE   rmse  20.037751"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    fcst_timegpt.merge(df_test, on=['unique_id', 'ds']),\n",
    "    metrics=metrics,\n",
    "    models=['TimeGPT']\n",
    ")\n",
    "evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 89156 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_series(df_sub.iloc[-150:], forecasts_df= fcst_timegpt, level = [90])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Methods to Improve Forecast Accuracy\n",
    "### 3a. Add Finetune Steps\n",
    "The first approach to enhance forecast accuracy is to increase the number of fine-tuning steps. The fine-tuning process adjusts the weights within the TimeGPT model, allowing it to better fit your customized data. This adjustment enables TimeGPT to learn the nuances of your time series more effectively, leading to more accurate forecasts. With 30 fine-tuning steps, we observe that the MAE decreases to 11.5 and the RMSE drops to 12.6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "fcst_finetune_df = nixtla_client.forecast(df=df_train[['unique_id', 'ds', 'y']],\n",
    "                                          h=24*2,\n",
    "                                          finetune_steps = 30,\n",
    "                                          level=[90, 95])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>metric</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DE</td>\n",
       "      <td>mae</td>\n",
       "      <td>11.458185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DE</td>\n",
       "      <td>rmse</td>\n",
       "      <td>12.642999</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id metric    TimeGPT\n",
       "0        DE    mae  11.458185\n",
       "1        DE   rmse  12.642999"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    fcst_finetune_df.merge(df_test, on=['unique_id', 'ds']),\n",
    "    metrics=metrics,\n",
    "    models=['TimeGPT']\n",
    ")\n",
    "evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 94496 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_series(df_sub[-200:], forecasts_df= fcst_finetune_df, level = [90])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3b. Finetune with Different Loss Function\n",
    "The second way to further reduce forecast error is to adjust the loss function used during fine-tuning. You can specify your customized loss function using the `finetune_loss` parameter. By modifying the loss function, we observe that the MAE decreases to 9.6 and the RMSE reduces to 11.0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "fcst_finetune_mae_df = nixtla_client.forecast(df=df_train[['unique_id', 'ds', 'y']],\n",
    "                                          h=24*2,\n",
    "                                          finetune_steps = 30,\n",
    "                                          finetune_loss = 'mae',\n",
    "                                          level=[90, 95])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>metric</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DE</td>\n",
       "      <td>mae</td>\n",
       "      <td>9.640649</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DE</td>\n",
       "      <td>rmse</td>\n",
       "      <td>10.956003</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id metric    TimeGPT\n",
       "0        DE    mae   9.640649\n",
       "1        DE   rmse  10.956003"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    fcst_finetune_mae_df.merge(df_test, on=['unique_id', 'ds']),\n",
    "    metrics=metrics,\n",
    "    models=['TimeGPT']\n",
    ")\n",
    "evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 94268 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_series(df_sub[-200:], forecasts_df= fcst_finetune_mae_df, level = [90])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3c. Adjust the number of parameters being fine-tuned\n",
    "\n",
    "Using the `finetune_depth` parameter, we can control the number of parameters that get fine-tuned. By default, `finetune_depth=1`, meaning that few parameters are tuned. We can set it to any value from 1 to 5, where 5 means that we fine-tune all of the parameters of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "fcst_finetune_depth_df = nixtla_client.forecast(df=df_train[['unique_id', 'ds', 'y']],\n",
    "                                                h=24*2,\n",
    "                                                finetune_steps = 30,\n",
    "                                                finetune_depth=2,\n",
    "                                                finetune_loss = 'mae',\n",
    "                                                level=[90, 95])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>metric</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DE</td>\n",
       "      <td>mae</td>\n",
       "      <td>9.002193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DE</td>\n",
       "      <td>rmse</td>\n",
       "      <td>11.348207</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id metric    TimeGPT\n",
       "0        DE    mae   9.002193\n",
       "1        DE   rmse  11.348207"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    fcst_finetune_depth_df.merge(df_test, on=['unique_id', 'ds']),\n",
    "    metrics=metrics,\n",
    "    models=['TimeGPT']\n",
    ")\n",
    "evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 91084 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_series(df_sub[-200:], forecasts_df= fcst_finetune_depth_df, level = [90])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3d. Forecast with Exogenous Variables\n",
    "Exogenous variables are external factors or predictors that are not part of the target time series but can influence its behavior. Incorporating these variables can provide the model with additional context, improving its ability to understand complex relationships and patterns in the data.\n",
    "\n",
    "To use exogenous variables in TimeGPT, pair each point in your input time series with the corresponding external data. If you have future values available for these variables during the forecast period, include them using the X_df parameter. Otherwise, you can omit this parameter and still see improvements using only historical values. In the example below, we incorporate 8 historical exogenous variables along with their values during the test period, which reduces the MAE and RMSE to 4.6 and 6.4, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>Exogenous1</th>\n",
       "      <th>Exogenous2</th>\n",
       "      <th>day_0</th>\n",
       "      <th>day_1</th>\n",
       "      <th>day_2</th>\n",
       "      <th>day_3</th>\n",
       "      <th>day_4</th>\n",
       "      <th>day_5</th>\n",
       "      <th>day_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1680</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-10-22 00:00:00</td>\n",
       "      <td>19.10</td>\n",
       "      <td>16972.75</td>\n",
       "      <td>15778.92975</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1681</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-10-22 01:00:00</td>\n",
       "      <td>19.03</td>\n",
       "      <td>16254.50</td>\n",
       "      <td>16664.20950</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1682</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-10-22 02:00:00</td>\n",
       "      <td>16.90</td>\n",
       "      <td>15940.25</td>\n",
       "      <td>17728.74950</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1683</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-10-22 03:00:00</td>\n",
       "      <td>12.98</td>\n",
       "      <td>15959.50</td>\n",
       "      <td>18578.13850</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1684</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-10-22 04:00:00</td>\n",
       "      <td>9.24</td>\n",
       "      <td>16071.50</td>\n",
       "      <td>19389.16750</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id                  ds      y  Exogenous1   Exogenous2  day_0  \\\n",
       "1680        DE 2017-10-22 00:00:00  19.10    16972.75  15778.92975    0.0   \n",
       "1681        DE 2017-10-22 01:00:00  19.03    16254.50  16664.20950    0.0   \n",
       "1682        DE 2017-10-22 02:00:00  16.90    15940.25  17728.74950    0.0   \n",
       "1683        DE 2017-10-22 03:00:00  12.98    15959.50  18578.13850    0.0   \n",
       "1684        DE 2017-10-22 04:00:00   9.24    16071.50  19389.16750    0.0   \n",
       "\n",
       "      day_1  day_2  day_3  day_4  day_5  day_6  \n",
       "1680    0.0    0.0    0.0    0.0    0.0    1.0  \n",
       "1681    0.0    0.0    0.0    0.0    0.0    1.0  \n",
       "1682    0.0    0.0    0.0    0.0    0.0    1.0  \n",
       "1683    0.0    0.0    0.0    0.0    0.0    1.0  \n",
       "1684    0.0    0.0    0.0    0.0    0.0    1.0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>Exogenous1</th>\n",
       "      <th>Exogenous2</th>\n",
       "      <th>day_0</th>\n",
       "      <th>day_1</th>\n",
       "      <th>day_2</th>\n",
       "      <th>day_3</th>\n",
       "      <th>day_4</th>\n",
       "      <th>day_5</th>\n",
       "      <th>day_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3312</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-12-29 00:00:00</td>\n",
       "      <td>17347.00</td>\n",
       "      <td>24577.92650</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3313</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-12-29 01:00:00</td>\n",
       "      <td>16587.25</td>\n",
       "      <td>24554.31950</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3314</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-12-29 02:00:00</td>\n",
       "      <td>16396.00</td>\n",
       "      <td>24651.45475</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3315</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-12-29 03:00:00</td>\n",
       "      <td>16481.25</td>\n",
       "      <td>24666.04300</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3316</th>\n",
       "      <td>DE</td>\n",
       "      <td>2017-12-29 04:00:00</td>\n",
       "      <td>16827.75</td>\n",
       "      <td>24403.33350</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id                  ds  Exogenous1   Exogenous2  day_0  day_1  \\\n",
       "3312        DE 2017-12-29 00:00:00    17347.00  24577.92650    0.0    0.0   \n",
       "3313        DE 2017-12-29 01:00:00    16587.25  24554.31950    0.0    0.0   \n",
       "3314        DE 2017-12-29 02:00:00    16396.00  24651.45475    0.0    0.0   \n",
       "3315        DE 2017-12-29 03:00:00    16481.25  24666.04300    0.0    0.0   \n",
       "3316        DE 2017-12-29 04:00:00    16827.75  24403.33350    0.0    0.0   \n",
       "\n",
       "      day_2  day_3  day_4  day_5  day_6  \n",
       "3312    0.0    0.0    1.0    0.0    0.0  \n",
       "3313    0.0    0.0    1.0    0.0    0.0  \n",
       "3314    0.0    0.0    1.0    0.0    0.0  \n",
       "3315    0.0    0.0    1.0    0.0    0.0  \n",
       "3316    0.0    0.0    1.0    0.0    0.0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "future_ex_vars_df = df_test.drop(columns = ['y'])\n",
    "future_ex_vars_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Using future exogenous features: ['Exogenous1', 'Exogenous2', 'day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6']\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "fcst_ex_vars_df = nixtla_client.forecast(df=df_train,\n",
    "                                         X_df=future_ex_vars_df,\n",
    "                                         h=24*2,\n",
    "                                         level=[90, 95])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>metric</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DE</td>\n",
       "      <td>mae</td>\n",
       "      <td>4.602594</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DE</td>\n",
       "      <td>rmse</td>\n",
       "      <td>6.358831</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id metric   TimeGPT\n",
       "0        DE    mae  4.602594\n",
       "1        DE   rmse  6.358831"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    fcst_ex_vars_df.merge(df_test, on=['unique_id', 'ds']),\n",
    "    metrics=metrics,\n",
    "    models=['TimeGPT']\n",
    ")\n",
    "evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 92396 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_series(df_sub[-200:], forecasts_df= fcst_ex_vars_df, level = [90])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3d. TimeGPT for Long Horizon Forecasting\n",
    "When the forecasting period is too long, the predicted results may not be as accurate. TimeGPT performs best with forecast periods that are shorter than one complete cycle of the time series. For longer forecast periods, switching to the timegpt-1-long-horizon model can yield better results. You can specify this model by using the model parameter.\n",
    "\n",
    "In the electricity price time series used here, one cycle is 24 steps (representing one day). Since we’re forecasting two days (48 steps) into the future, using timegpt-1-long-horizon significantly improves the forecasting accuracy, reducing the MAE to 6.4 and RMSE to 7.7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Querying model metadata...\n",
      "INFO:nixtla.nixtla_client:Restricting input...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "fcst_long_df = nixtla_client.forecast(df=df_train[['unique_id', 'ds', 'y']],\n",
    "                                          h=24*2,\n",
    "                                          model = 'timegpt-1-long-horizon',\n",
    "                                          level=[90, 95])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>metric</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DE</td>\n",
       "      <td>mae</td>\n",
       "      <td>6.365540</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DE</td>\n",
       "      <td>rmse</td>\n",
       "      <td>7.738188</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id metric   TimeGPT\n",
       "0        DE    mae  6.365540\n",
       "1        DE   rmse  7.738188"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    fcst_long_df.merge(df_test, on=['unique_id', 'ds']),\n",
    "    metrics=metrics,\n",
    "    models=['TimeGPT']\n",
    ")\n",
    "evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 95596 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_series(df_sub[-200:], forecasts_df= fcst_long_df, level = [90])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Conclusion and Next Steps\n",
    "\n",
    "In this notebook, we demonstrated four effective strategies for enhancing forecast accuracy with TimeGPT:\n",
    "\n",
    "1. **Increasing the number of fine-tuning steps.**\n",
    "2. **Adjusting the fine-tuning loss function.**\n",
    "3. **Incorporating exogenous variables.**\n",
    "4. **Switching to the long-horizon model for extended forecasting periods.**\n",
    "\n",
    "We encourage you to experiment with these hyperparameters to identify the optimal settings that best suit your specific needs. Additionally, please refer to our documentation for further features, such as **model explainability** and more.\n",
    "\n",
    "In the examples provided, after applying these methods, we observed significant improvements in forecast accuracy metrics, as summarized below.\n",
    "\n",
    "### Result Summary\n",
    "\n",
    "| Steps | Description                  | MAE  | MAE Improvement (%) | RMSE  | RMSE Improvement (%) |\n",
    "|-------|------------------------------|------|---------------------|-------|----------------------|\n",
    "| 0     | Zero-Shot TimeGPT            | 18.5 | N/A                 | 20.0  | N/A                  |\n",
    "| 1     | Add Fine-Tuning Steps        | 11.5 | 38%                 | 12.6  | 37%                  |\n",
    "| 2     | Adjust Fine-Tuning Loss      | 9.6  | 48%                 | 11.0  | 45%                  |\n",
    "| 3     | Fine-tune more parameters    | 9.0  | 51%                 | 11.3  | 44%                  |\n",
    "| 4     | Add Exogenous Variables      | 4.6  | 75%                 | 6.4   | 68%                  |\n",
    "| 5     | Switch to Long-Horizon Model | 6.4  | 65%                 | 7.7   | 62%                  |"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
