{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "!pip install -Uqq nixtla utilsforecast statsforecast"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "from nixtla.utils import in_colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "IN_COLAB = in_colab()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    from nixtla.utils import colab_badge\n",
    "    from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Forecasting Intermittent Demand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial, we show how to use TimeGPT on an intermittent series where we have many values at zero. Here, we use a subset of the M5 dataset that tracks the demand for food items in a Californian store. The dataset also includes exogenous variables like the sell price and the type of event occuring at a particular day.\n",
    "\n",
    "TimeGPT achieves the best performance at a MAE of 0.49, which represents a **14% improvement** over the best statistical model specifically built to handle intermittent time series data.\n",
    "\n",
    "Predicting with TimeGPT took 6.8 seconds, while fitting and predicting with statistical models took 5.2 seconds. TimeGPT is technically slower, but for a difference in time of roughly 1 second only, we get much better predictions with TimeGPT."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/tutorials/4_intermittent_demand.ipynb)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#| echo: false\n",
    "if not IN_COLAB:\n",
    "    load_dotenv()\n",
    "    colab_badge('docs/tutorials/4_intermittent_demand')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initial setup\n",
    "\n",
    "We start off by importing the required packages for this tutorial and create an instace of `NixtlaClient`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from nixtla import NixtlaClient\n",
    "\n",
    "from utilsforecast.losses import mae\n",
    "from utilsforecast.evaluation import evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nixtla_client = NixtlaClient(\n",
    "    # defaults to os.environ.get(\"NIXTLA_API_KEY\")\n",
    "    api_key = 'my_api_key_provided_by_nixtla'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 👍 Use an Azure AI endpoint\n",
    "> \n",
    "> To use an Azure AI endpoint, remember to set also the `base_url` argument:\n",
    "> \n",
    "> `nixtla_client = NixtlaClient(base_url=\"you azure ai endpoint\", api_key=\"your api_key\")`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    nixtla_client = NixtlaClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now read the dataset and plot it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>sell_price</th>\n",
       "      <th>event_type_Cultural</th>\n",
       "      <th>event_type_National</th>\n",
       "      <th>event_type_Religious</th>\n",
       "      <th>event_type_Sporting</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-01-29</td>\n",
       "      <td>3</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-01-30</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-01-31</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-02-01</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-02-02</td>\n",
       "      <td>4</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id         ds  y  sell_price  event_type_Cultural  \\\n",
       "0  FOODS_1_001 2011-01-29  3         2.0                    0   \n",
       "1  FOODS_1_001 2011-01-30  0         2.0                    0   \n",
       "2  FOODS_1_001 2011-01-31  0         2.0                    0   \n",
       "3  FOODS_1_001 2011-02-01  1         2.0                    0   \n",
       "4  FOODS_1_001 2011-02-02  4         2.0                    0   \n",
       "\n",
       "   event_type_National  event_type_Religious  event_type_Sporting  \n",
       "0                    0                     0                    0  \n",
       "1                    0                     0                    0  \n",
       "2                    0                     0                    0  \n",
       "3                    0                     0                    0  \n",
       "4                    0                     0                    0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/m5_sales_exog_small.csv\")\n",
    "df['ds'] = pd.to_datetime(df['ds'])\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 529636 characters]",
      "text/plain": [
       "<Figure size 2400x1400 with 8 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df, \n",
    "    max_insample_length=365, \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the figure above, we can see the intermittent nature of this dataset, with many periods with zero demand.\n",
    "\n",
    "Now, let's use TimeGPT to forecast the demand of each product."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bounded forecasts\n",
    "\n",
    "To avoid getting negative predictions coming from the model, we use a log transformation on the data. That way, the model will be forced to predict only positive values.\n",
    "\n",
    "Note that due to the presence of zeros in our dataset, we add one to all points before taking the log."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>sell_price</th>\n",
       "      <th>event_type_Cultural</th>\n",
       "      <th>event_type_National</th>\n",
       "      <th>event_type_Religious</th>\n",
       "      <th>event_type_Sporting</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-01-29</td>\n",
       "      <td>1.386294</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-01-30</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-01-31</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-02-01</td>\n",
       "      <td>0.693147</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2011-02-02</td>\n",
       "      <td>1.609438</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id         ds         y  sell_price  event_type_Cultural  \\\n",
       "0  FOODS_1_001 2011-01-29  1.386294         2.0                    0   \n",
       "1  FOODS_1_001 2011-01-30  0.000000         2.0                    0   \n",
       "2  FOODS_1_001 2011-01-31  0.000000         2.0                    0   \n",
       "3  FOODS_1_001 2011-02-01  0.693147         2.0                    0   \n",
       "4  FOODS_1_001 2011-02-02  1.609438         2.0                    0   \n",
       "\n",
       "   event_type_National  event_type_Religious  event_type_Sporting  \n",
       "0                    0                     0                    0  \n",
       "1                    0                     0                    0  \n",
       "2                    0                     0                    0  \n",
       "3                    0                     0                    0  \n",
       "4                    0                     0                    0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_transformed = df.copy()\n",
    "\n",
    "df_transformed['y'] = np.log(df_transformed['y']+1)\n",
    "\n",
    "df_transformed.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's keep the last 28 time steps for the test set and use the rest as input to the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_df = df_transformed.groupby('unique_id').tail(28)                                                      \n",
    "\n",
    "input_df = df_transformed.drop(test_df.index).reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Forecasting with TimeGPT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: D\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time (TimeGPT): 6.164413213729858\n"
     ]
    }
   ],
   "source": [
    "start = time.time()\n",
    "\n",
    "fcst_df = nixtla_client.forecast(\n",
    "    df=input_df,\n",
    "    h=28,                            \n",
    "    level=[80],                        # Generate a 80% confidence interval\n",
    "    finetune_steps=10,                 # Specify the number of steps for fine-tuning\n",
    "    finetune_loss='mae',               # Use the MAE as the loss function for fine-tuning\n",
    "    model='timegpt-1-long-horizon',    # Use the model for long-horizon forecasting\n",
    "    time_col='ds',\n",
    "    target_col='y',\n",
    "    id_col='unique_id'\n",
    ")\n",
    "\n",
    "end = time.time()\n",
    "\n",
    "timegpt_duration = end - start\n",
    "\n",
    "print(f\"Time (TimeGPT): {timegpt_duration}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! TimeGPT was done in **5.8 seconds** and we now have predictions. However, those predictions are transformed, so we need to inverse the transformation to get back to the orignal scale. Therefore, we take the exponential and subtract one from each data point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-23</td>\n",
       "      <td>0.286841</td>\n",
       "      <td>-0.267101</td>\n",
       "      <td>1.259465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-24</td>\n",
       "      <td>0.320482</td>\n",
       "      <td>-0.241236</td>\n",
       "      <td>1.298046</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-25</td>\n",
       "      <td>0.287392</td>\n",
       "      <td>-0.362250</td>\n",
       "      <td>1.598791</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-26</td>\n",
       "      <td>0.295326</td>\n",
       "      <td>-0.145489</td>\n",
       "      <td>0.963542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-27</td>\n",
       "      <td>0.315868</td>\n",
       "      <td>-0.166516</td>\n",
       "      <td>1.077437</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id          ds   TimeGPT  TimeGPT-lo-80  TimeGPT-hi-80\n",
       "0  FOODS_1_001  2016-05-23  0.286841      -0.267101       1.259465\n",
       "1  FOODS_1_001  2016-05-24  0.320482      -0.241236       1.298046\n",
       "2  FOODS_1_001  2016-05-25  0.287392      -0.362250       1.598791\n",
       "3  FOODS_1_001  2016-05-26  0.295326      -0.145489       0.963542\n",
       "4  FOODS_1_001  2016-05-27  0.315868      -0.166516       1.077437"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols = [col for col in fcst_df.columns if col not in ['ds', 'unique_id']]\n",
    "\n",
    "for col in cols:\n",
    "    fcst_df[col] = np.exp(fcst_df[col])-1\n",
    "\n",
    "fcst_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before measuring the performance metric, let's plot the predictions against the actual values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 498760 characters]",
      "text/plain": [
       "<Figure size 2400x1400 with 8 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(test_df, fcst_df, models=['TimeGPT'], level=[80], time_col='ds', target_col='y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can measure the mean absolute error (MAE) of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fcst_df['ds'] = pd.to_datetime(fcst_df['ds'])\n",
    "\n",
    "test_df = pd.merge(test_df, fcst_df, 'left', ['unique_id', 'ds'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "metric\n",
       "mae    0.492559\n",
       "Name: TimeGPT, dtype: float64"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    test_df,\n",
    "    metrics=[mae],\n",
    "    models=[\"TimeGPT\"],\n",
    "    target_col=\"y\",\n",
    "    id_col='unique_id'\n",
    ")\n",
    "\n",
    "average_metrics = evaluation.groupby('metric')['TimeGPT'].mean()\n",
    "average_metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Forecasting with statistical models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The library `statsforecast` by Nixtla provides a suite of statistical models specifically built for intermittent forecasting, such as Croston, IMAPA and TSB. Let's use these models and see how they perform against TimeGPT."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from statsforecast import StatsForecast\n",
    "from statsforecast.models import CrostonClassic, CrostonOptimized, IMAPA, TSB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we use four models: two versions of Croston, IMAPA and TSB."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "models = [CrostonClassic(), CrostonOptimized(), IMAPA(), TSB(0.1, 0.1)]\n",
    "\n",
    "sf = StatsForecast(\n",
    "    models=models,\n",
    "    freq='D',\n",
    "    n_jobs=-1\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we can fit the models on our data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "start = time.time()\n",
    "\n",
    "sf.fit(df=input_df)\n",
    "\n",
    "sf_preds = sf.predict(h=28)\n",
    "\n",
    "end = time.time()\n",
    "\n",
    "sf_duration = end - start\n",
    "\n",
    "print(f\"Statistical models took :{sf_duration}s\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, fitting and predicting with four statistical models took 5.2 seconds, while TimeGPT took 5.8 seconds, so TimeGPT was only 0.6 seconds slower."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, we need to inverse the transformation. Remember that the training data was previously transformed using the log function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ds</th>\n",
       "      <th>CrostonClassic</th>\n",
       "      <th>CrostonOptimized</th>\n",
       "      <th>IMAPA</th>\n",
       "      <th>TSB</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>FOODS_1_001</th>\n",
       "      <td>2016-05-23</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FOODS_1_001</th>\n",
       "      <td>2016-05-24</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FOODS_1_001</th>\n",
       "      <td>2016-05-25</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FOODS_1_001</th>\n",
       "      <td>2016-05-26</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FOODS_1_001</th>\n",
       "      <td>2016-05-27</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    ds  CrostonClassic  CrostonOptimized     IMAPA       TSB\n",
       "unique_id                                                                   \n",
       "FOODS_1_001 2016-05-23        0.599093          0.599093  0.445779  0.396258\n",
       "FOODS_1_001 2016-05-24        0.599093          0.599093  0.445779  0.396258\n",
       "FOODS_1_001 2016-05-25        0.599093          0.599093  0.445779  0.396258\n",
       "FOODS_1_001 2016-05-26        0.599093          0.599093  0.445779  0.396258\n",
       "FOODS_1_001 2016-05-27        0.599093          0.599093  0.445779  0.396258"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols = [col for col in sf_preds.columns if col not in ['ds', 'unique_id']]\n",
    "\n",
    "for col in cols:\n",
    "    sf_preds[col] = np.exp(sf_preds[col])-1\n",
    "\n",
    "sf_preds.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's combine the predictions from all methods and see which performs best."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>sell_price</th>\n",
       "      <th>event_type_Cultural</th>\n",
       "      <th>event_type_National</th>\n",
       "      <th>event_type_Religious</th>\n",
       "      <th>event_type_Sporting</th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "      <th>CrostonClassic</th>\n",
       "      <th>CrostonOptimized</th>\n",
       "      <th>IMAPA</th>\n",
       "      <th>TSB</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-23</td>\n",
       "      <td>1.386294</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.286841</td>\n",
       "      <td>-0.267101</td>\n",
       "      <td>1.259465</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-24</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.320482</td>\n",
       "      <td>-0.241236</td>\n",
       "      <td>1.298046</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-25</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.287392</td>\n",
       "      <td>-0.362250</td>\n",
       "      <td>1.598791</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-26</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.295326</td>\n",
       "      <td>-0.145489</td>\n",
       "      <td>0.963542</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-27</td>\n",
       "      <td>1.945910</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.315868</td>\n",
       "      <td>-0.166516</td>\n",
       "      <td>1.077437</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id         ds         y  sell_price  event_type_Cultural  \\\n",
       "0  FOODS_1_001 2016-05-23  1.386294        2.24                    0   \n",
       "1  FOODS_1_001 2016-05-24  0.000000        2.24                    0   \n",
       "2  FOODS_1_001 2016-05-25  0.000000        2.24                    0   \n",
       "3  FOODS_1_001 2016-05-26  0.000000        2.24                    0   \n",
       "4  FOODS_1_001 2016-05-27  1.945910        2.24                    0   \n",
       "\n",
       "   event_type_National  event_type_Religious  event_type_Sporting   TimeGPT  \\\n",
       "0                    0                     0                    0  0.286841   \n",
       "1                    0                     0                    0  0.320482   \n",
       "2                    0                     0                    0  0.287392   \n",
       "3                    0                     0                    0  0.295326   \n",
       "4                    0                     0                    0  0.315868   \n",
       "\n",
       "   TimeGPT-lo-80  TimeGPT-hi-80  CrostonClassic  CrostonOptimized     IMAPA  \\\n",
       "0      -0.267101       1.259465        0.599093          0.599093  0.445779   \n",
       "1      -0.241236       1.298046        0.599093          0.599093  0.445779   \n",
       "2      -0.362250       1.598791        0.599093          0.599093  0.445779   \n",
       "3      -0.145489       0.963542        0.599093          0.599093  0.445779   \n",
       "4      -0.166516       1.077437        0.599093          0.599093  0.445779   \n",
       "\n",
       "        TSB  \n",
       "0  0.396258  \n",
       "1  0.396258  \n",
       "2  0.396258  \n",
       "3  0.396258  \n",
       "4  0.396258  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.merge(test_df, sf_preds, 'left', ['unique_id', 'ds'])\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>CrostonClassic</th>\n",
       "      <th>CrostonOptimized</th>\n",
       "      <th>IMAPA</th>\n",
       "      <th>TSB</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>metric</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>mae</th>\n",
       "      <td>0.492559</td>\n",
       "      <td>0.564563</td>\n",
       "      <td>0.580922</td>\n",
       "      <td>0.571943</td>\n",
       "      <td>0.567178</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         TimeGPT  CrostonClassic  CrostonOptimized     IMAPA       TSB\n",
       "metric                                                                \n",
       "mae     0.492559        0.564563          0.580922  0.571943  0.567178"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    test_df,\n",
    "    metrics=[mae],\n",
    "    models=[\"TimeGPT\", \"CrostonClassic\", \"CrostonOptimized\", \"IMAPA\", \"TSB\"],\n",
    "    target_col=\"y\",\n",
    "    id_col='unique_id'\n",
    ")\n",
    "\n",
    "average_metrics = evaluation.groupby('metric')[[\"TimeGPT\", \"CrostonClassic\", \"CrostonOptimized\", \"IMAPA\", \"TSB\"]].mean()\n",
    "average_metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the table above, we can see that TimeGPT achieves the lowest MAE, achieving a 12.8% improvement over the best performing statistical model.\n",
    "\n",
    "Now, this was done without using any of the available exogenous features. While the statsitical models do not support them, let's try including them in TimeGPT."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Forecasting with exogenous variables using TimeGPT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To forecast with exogenous variables, we need to specify their future values over the forecast horizon. Therefore, let's simply take the types of events, as those dates are known in advance. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>event_type_Cultural</th>\n",
       "      <th>event_type_National</th>\n",
       "      <th>event_type_Religious</th>\n",
       "      <th>event_type_Sporting</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-23</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-25</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id         ds  event_type_Cultural  event_type_National  \\\n",
       "0  FOODS_1_001 2016-05-23                    0                    0   \n",
       "1  FOODS_1_001 2016-05-24                    0                    0   \n",
       "2  FOODS_1_001 2016-05-25                    0                    0   \n",
       "3  FOODS_1_001 2016-05-26                    0                    0   \n",
       "4  FOODS_1_001 2016-05-27                    0                    0   \n",
       "\n",
       "   event_type_Religious  event_type_Sporting  \n",
       "0                     0                    0  \n",
       "1                     0                    0  \n",
       "2                     0                    0  \n",
       "3                     0                    0  \n",
       "4                     0                    0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "futr_exog_df = test_df.drop([\"TimeGPT\", \"CrostonClassic\", \"CrostonOptimized\", \"IMAPA\", \"TSB\", \"y\", \"TimeGPT-lo-80\", \"TimeGPT-hi-80\", \"sell_price\"], axis=1)\n",
    "futr_exog_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we simply call the `forecast` method and pass the `futr_exog_df` in the `X_df` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: D\n",
      "INFO:nixtla.nixtla_client:Using the following exogenous variables: event_type_Cultural, event_type_National, event_type_Religious, event_type_Sporting\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time (TimeGPT): 7.173351287841797\n"
     ]
    }
   ],
   "source": [
    "start = time.time()\n",
    "\n",
    "fcst_df = nixtla_client.forecast(\n",
    "    df=input_df,\n",
    "    X_df=futr_exog_df,\n",
    "    h=28,                            \n",
    "    level=[80],                        # Generate a 80% confidence interval\n",
    "    finetune_steps=10,                 # Specify the number of steps for fine-tuning\n",
    "    finetune_loss='mae',               # Use the MAE as the loss function for fine-tuning\n",
    "    model='timegpt-1-long-horizon',    # Use the model for long-horizon forecasting\n",
    "    time_col='ds',\n",
    "    target_col='y',\n",
    "    id_col='unique_id'\n",
    ")\n",
    "\n",
    "end = time.time()\n",
    "\n",
    "timegpt_duration = end - start\n",
    "\n",
    "print(f\"Time (TimeGPT): {timegpt_duration}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Remember that the predictions are transformed, so we have to inverse the transformation again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT_ex</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-23</td>\n",
       "      <td>0.281922</td>\n",
       "      <td>-0.269902</td>\n",
       "      <td>1.250828</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-24</td>\n",
       "      <td>0.313774</td>\n",
       "      <td>-0.245091</td>\n",
       "      <td>1.286372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-25</td>\n",
       "      <td>0.285639</td>\n",
       "      <td>-0.363119</td>\n",
       "      <td>1.595252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-26</td>\n",
       "      <td>0.295037</td>\n",
       "      <td>-0.145679</td>\n",
       "      <td>0.963104</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-27</td>\n",
       "      <td>0.315484</td>\n",
       "      <td>-0.166760</td>\n",
       "      <td>1.076830</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id          ds  TimeGPT_ex  TimeGPT-lo-80  TimeGPT-hi-80\n",
       "0  FOODS_1_001  2016-05-23    0.281922      -0.269902       1.250828\n",
       "1  FOODS_1_001  2016-05-24    0.313774      -0.245091       1.286372\n",
       "2  FOODS_1_001  2016-05-25    0.285639      -0.363119       1.595252\n",
       "3  FOODS_1_001  2016-05-26    0.295037      -0.145679       0.963104\n",
       "4  FOODS_1_001  2016-05-27    0.315484      -0.166760       1.076830"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fcst_df.rename(columns={\n",
    "    'TimeGPT': 'TimeGPT_ex',\n",
    "}, inplace=True)\n",
    "\n",
    "cols = [col for col in fcst_df.columns if col not in ['ds', 'unique_id']]\n",
    "\n",
    "for col in cols:\n",
    "    fcst_df[col] = np.exp(fcst_df[col])-1\n",
    "\n",
    "fcst_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's evaluate the performance of TimeGPT with exogenous features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>sell_price</th>\n",
       "      <th>event_type_Cultural</th>\n",
       "      <th>event_type_National</th>\n",
       "      <th>event_type_Religious</th>\n",
       "      <th>event_type_Sporting</th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "      <th>CrostonClassic</th>\n",
       "      <th>CrostonOptimized</th>\n",
       "      <th>IMAPA</th>\n",
       "      <th>TSB</th>\n",
       "      <th>TimeGPT_ex</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-23</td>\n",
       "      <td>1.386294</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.286841</td>\n",
       "      <td>-0.267101</td>\n",
       "      <td>1.259465</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "      <td>0.281922</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-24</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.320482</td>\n",
       "      <td>-0.241236</td>\n",
       "      <td>1.298046</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "      <td>0.313774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-25</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.287392</td>\n",
       "      <td>-0.362250</td>\n",
       "      <td>1.598791</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "      <td>0.285639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-26</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.295326</td>\n",
       "      <td>-0.145489</td>\n",
       "      <td>0.963542</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "      <td>0.295037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001</td>\n",
       "      <td>2016-05-27</td>\n",
       "      <td>1.945910</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.315868</td>\n",
       "      <td>-0.166516</td>\n",
       "      <td>1.077437</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.599093</td>\n",
       "      <td>0.445779</td>\n",
       "      <td>0.396258</td>\n",
       "      <td>0.315484</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id         ds         y  sell_price  event_type_Cultural  \\\n",
       "0  FOODS_1_001 2016-05-23  1.386294        2.24                    0   \n",
       "1  FOODS_1_001 2016-05-24  0.000000        2.24                    0   \n",
       "2  FOODS_1_001 2016-05-25  0.000000        2.24                    0   \n",
       "3  FOODS_1_001 2016-05-26  0.000000        2.24                    0   \n",
       "4  FOODS_1_001 2016-05-27  1.945910        2.24                    0   \n",
       "\n",
       "   event_type_National  event_type_Religious  event_type_Sporting   TimeGPT  \\\n",
       "0                    0                     0                    0  0.286841   \n",
       "1                    0                     0                    0  0.320482   \n",
       "2                    0                     0                    0  0.287392   \n",
       "3                    0                     0                    0  0.295326   \n",
       "4                    0                     0                    0  0.315868   \n",
       "\n",
       "   TimeGPT-lo-80  TimeGPT-hi-80  CrostonClassic  CrostonOptimized     IMAPA  \\\n",
       "0      -0.267101       1.259465        0.599093          0.599093  0.445779   \n",
       "1      -0.241236       1.298046        0.599093          0.599093  0.445779   \n",
       "2      -0.362250       1.598791        0.599093          0.599093  0.445779   \n",
       "3      -0.145489       0.963542        0.599093          0.599093  0.445779   \n",
       "4      -0.166516       1.077437        0.599093          0.599093  0.445779   \n",
       "\n",
       "        TSB  TimeGPT_ex  \n",
       "0  0.396258    0.281922  \n",
       "1  0.396258    0.313774  \n",
       "2  0.396258    0.285639  \n",
       "3  0.396258    0.295037  \n",
       "4  0.396258    0.315484  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['TimeGPT_ex'] = fcst_df['TimeGPT_ex'].values\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>CrostonClassic</th>\n",
       "      <th>CrostonOptimized</th>\n",
       "      <th>IMAPA</th>\n",
       "      <th>TSB</th>\n",
       "      <th>TimeGPT_ex</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>metric</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>mae</th>\n",
       "      <td>0.492559</td>\n",
       "      <td>0.564563</td>\n",
       "      <td>0.580922</td>\n",
       "      <td>0.571943</td>\n",
       "      <td>0.567178</td>\n",
       "      <td>0.485352</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         TimeGPT  CrostonClassic  CrostonOptimized     IMAPA       TSB  \\\n",
       "metric                                                                   \n",
       "mae     0.492559        0.564563          0.580922  0.571943  0.567178   \n",
       "\n",
       "        TimeGPT_ex  \n",
       "metric              \n",
       "mae       0.485352  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluation = evaluate(\n",
    "    test_df,\n",
    "    metrics=[mae],\n",
    "    models=[\"TimeGPT\", \"CrostonClassic\", \"CrostonOptimized\", \"IMAPA\", \"TSB\", \"TimeGPT_ex\"],\n",
    "    target_col=\"y\",\n",
    "    id_col='unique_id'\n",
    ")\n",
    "\n",
    "average_metrics = evaluation.groupby('metric')[[\"TimeGPT\", \"CrostonClassic\", \"CrostonOptimized\", \"IMAPA\", \"TSB\", \"TimeGPT_ex\"]].mean()\n",
    "average_metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the table above, we can see that using exogenous features improved the performance of TimeGPT. Now, it represents a 14% improvement over the best statistical model. \n",
    "\n",
    "Using TimeGPT with exogenous features took 6.8 seconds. This is 1.6 seconds slower than statitstical models, but it resulted in much better predictions."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
