{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3.10/pty.py:89: RuntimeWarning: Using fork() can cause Polars to deadlock in the child process.\n",
      "In addition, using fork() with Python in general is a recipe for mysterious\n",
      "deadlocks and crashes.\n",
      "\n",
      "The most likely reason you are seeing this error is because you are using the\n",
      "multiprocessing module on Linux, which uses fork() by default. This will be\n",
      "fixed in Python 3.14. Until then, you want to use the \"spawn\" context instead.\n",
      "\n",
      "See https://docs.pola.rs/user-guide/misc/multiprocessing/ for details.\n",
      "\n",
      "If you really know what your doing, you can silence this warning with the warning module\n",
      "or by setting POLARS_ALLOW_FORKING_THREAD=1.\n",
      "\n",
      "  pid, fd = os.forkpty()\n"
     ]
    }
   ],
   "source": [
    "#| hide\n",
    "!pip install -Uqq nixtla nbdev"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "from nixtla.utils import in_colab\n",
    "from nbdev import show_doc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "IN_COLAB = in_colab()\n",
    "if not IN_COLAB:\n",
    "    from nixtla.utils import colab_badge\n",
    "    from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Missing Values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`TimeGPT` requires time series data that doesn't have any missing values. It is possible to have multiple series that begin and end on different dates, but it is essential that each series contains uninterrupted data for its given time frame.\n",
    "\n",
    "In this tutorial, we will show you how to deal with missing values in `TimeGPT`. \n",
    "\n",
    "**Outline** \n",
    "\n",
    "1. [Load Data](#load-data) \n",
    "\n",
    "2. [Get Started with TimeGPT](#get-started-with-timegpt)\n",
    "\n",
    "3. [Visualize Data](#visualize-data)\n",
    "\n",
    "4. [Fill Missing Values](#fill-missing-values)\n",
    "\n",
    "5. [Forecast with TimeGPT](#forecast-with-timegpt)\n",
    "\n",
    "6. [Important Considerations](#important-considerations)\n",
    "\n",
    "7. [References](#references)\n",
    "\n",
    "This work is based on skforecast's [Forecasting Time Series with Missing Values](https://cienciadedatos.net/documentos/py46-forecasting-time-series-missing-values) tutorial. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/tutorials/15_missing_values.ipynb)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#| echo: false\n",
    "if not IN_COLAB:\n",
    "    load_dotenv()\n",
    "    colab_badge('docs/tutorials/15_missing_values')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will first load the data using `pandas`. This dataset represents the daily number of bike rentals in a city. The column names are in Spanish, so we will rename them to `ds` for the dates and `y` for the number of bike rentals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2014-06-23</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2014-06-24</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2014-06-25</td>\n",
       "      <td>119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2014-06-26</td>\n",
       "      <td>135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2014-06-27</td>\n",
       "      <td>149</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           ds    y\n",
       "0  2014-06-23   99\n",
       "1  2014-06-24   72\n",
       "2  2014-06-25  119\n",
       "3  2014-06-26  135\n",
       "4  2014-06-27  149"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/JoaquinAmatRodrigo/Estadistica-machine-learning-python/master/data/usuarios_diarios_bicimad.csv')\n",
    "df = df[['fecha', 'Usos bicis total día']] # select date and target variable\n",
    "df.rename(columns={'fecha': 'ds', 'Usos bicis total día': 'y'}, inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For convenience, we will convert the dates to timestamps and assign a unique id to the series. Although we only have one series in this example, when dealing with multiple series, it is necessary to assign a unique id to each one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['ds'] = pd.to_datetime(df['ds'])\n",
    "df['unique_id'] = 'id1'\n",
    "df = df[['unique_id', 'ds', 'y']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will separate the data in a training and a test set. We will use the last 93 days as the test set. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_df = df[:-93]\n",
    "test_df = df[-93:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now introduce some missing values in the training set to demonstrate how to deal with them. This will be done as in the [skforecast](https://cienciadedatos.net/documentos/py46-forecasting-time-series-missing-values) tutorial. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask = ~((train_df['ds'] >= '2020-09-01') & (train_df['ds'] <= '2020-10-10')) &  ~((train_df['ds'] >= '2020-11-08') & (train_df['ds'] <= '2020-12-15'))\n",
    "\n",
    "train_df_gaps = train_df[mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get Started with TimeGPT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before proceeding, we will instantiate the `NixtlaClient` class, which provides access to all the methods from `TimeGPT`. To do this, you will need a Nixtla API key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nixtla import NixtlaClient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nixtla_client = NixtlaClient(\n",
    "    # defaults to os.environ.get(\"NIXTLA_API_KEY\")\n",
    "    api_key = 'my_api_key_provided_by_nixtla'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 👍 Use an Azure AI endpoint\n",
    ">\n",
    "> To use an Azure AI endpoint, set the `base_url` argument:\n",
    ">\n",
    "> `nixtla_client = NixtlaClient(base_url=\"you azure ai endpoint\", api_key=\"your api_key\")`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    nixtla_client = NixtlaClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To learn more about how to set up your API key, please refer to the [Setting Up Your API Key](https://docs.nixtla.io/docs/getting-started-setting_up_your_api_key) tutorial. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can visualize the data using the `plot` method from the `NixtlaClient` class. This method has an `engine` argument that allows you to choose between different plotting libraries. Default is `matplotlib`, but you can also use `plotly` for interactive plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 144308 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(train_df_gaps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that there are two gaps in the data: from September 1, 2020, to October 10, 2020, and from November 8, 2020, to December 15, 2020. To better visualize these gaps, you can use the `max_insample_length` argument of the `plot` method or you can simply zoom in on the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 139488 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(train_df_gaps, max_insample_length=800)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additionally, notice a period from March 16, 2020, to April 21, 2020, where the data shows zero rentals. These are not missing values, but actual zeros corresponding to the COVID-19 lockdown in the city."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fill Missing Values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before using `TimeGPT`, we need to ensure that: \n",
    "\n",
    "1. All timestamps from the start date to the end date are present in the data. \n",
    "\n",
    "2. The target column contains no missing values. \n",
    "\n",
    "To address the first issue, we will use the `fill_gaps` function from [`utilsforecast`](https://nixtlaverse.nixtla.io/utilsforecast/index.html), a Python package from Nixtla that provides essential utilities for time series forecasting, such as functions for data preprocessing, plotting, and evaluation.\n",
    "\n",
    "The `fill_gaps` function will fill in the missing dates in the data. To do this, it requires the following arguments:\n",
    "\n",
    "- `df`: The DataFrame containing the time series data.\n",
    "\n",
    "- `freq` (str or int): The frequency of the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from utilsforecast.preprocessing import fill_gaps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of rows before filling gaps: 2851\n",
      "Number of rows after filling gaps: 2929\n"
     ]
    }
   ],
   "source": [
    "print('Number of rows before filling gaps:', len(train_df_gaps))\n",
    "train_df_complete = fill_gaps(train_df_gaps, freq='D')\n",
    "print('Number of rows after filling gaps:', len(train_df_complete))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> NOTE:\n",
    ">\n",
    "> In this tutorial, the data contains only one time series. However, TimeGPT supports passing multiple series to the model. In this case, none of the time series can have missing values from their individual earliest timestamp until their individual lastest timestamp. If these individual time series have missing values, the user must decide how to fill these gaps for the individual time series. The [`fill_gaps`](https://nixtlaverse.nixtla.io/utilsforecast/preprocessing.html#fill-gaps) function provides a couple of additional arguments to assist with this (refer to the documentation for complete details), namely `start` and `end`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'mlforecast._modidx'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/IPython/core/formatters.py:770\u001b[0m, in \u001b[0;36mPlainTextFormatter.__call__\u001b[0;34m(self, obj)\u001b[0m\n\u001b[1;32m    763\u001b[0m stream \u001b[38;5;241m=\u001b[39m StringIO()\n\u001b[1;32m    764\u001b[0m printer \u001b[38;5;241m=\u001b[39m pretty\u001b[38;5;241m.\u001b[39mRepresentationPrinter(stream, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mverbose,\n\u001b[1;32m    765\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmax_width, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mnewline,\n\u001b[1;32m    766\u001b[0m     max_seq_length\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmax_seq_length,\n\u001b[1;32m    767\u001b[0m     singleton_pprinters\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39msingleton_printers,\n\u001b[1;32m    768\u001b[0m     type_pprinters\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtype_printers,\n\u001b[1;32m    769\u001b[0m     deferred_pprinters\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mdeferred_printers)\n\u001b[0;32m--> 770\u001b[0m \u001b[43mprinter\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpretty\u001b[49m\u001b[43m(\u001b[49m\u001b[43mobj\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    771\u001b[0m printer\u001b[38;5;241m.\u001b[39mflush()\n\u001b[1;32m    772\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m stream\u001b[38;5;241m.\u001b[39mgetvalue()\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/IPython/lib/pretty.py:419\u001b[0m, in \u001b[0;36mRepresentationPrinter.pretty\u001b[0;34m(self, obj)\u001b[0m\n\u001b[1;32m    408\u001b[0m                         \u001b[38;5;28;01mreturn\u001b[39;00m meth(obj, \u001b[38;5;28mself\u001b[39m, cycle)\n\u001b[1;32m    409\u001b[0m                 \u001b[38;5;28;01mif\u001b[39;00m (\n\u001b[1;32m    410\u001b[0m                     \u001b[38;5;28mcls\u001b[39m \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mobject\u001b[39m\n\u001b[1;32m    411\u001b[0m                     \u001b[38;5;66;03m# check if cls defines __repr__\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    417\u001b[0m                     \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mcallable\u001b[39m(_safe_getattr(\u001b[38;5;28mcls\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m__repr__\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mNone\u001b[39;00m))\n\u001b[1;32m    418\u001b[0m                 ):\n\u001b[0;32m--> 419\u001b[0m                     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43m_repr_pprint\u001b[49m\u001b[43m(\u001b[49m\u001b[43mobj\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcycle\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    421\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m _default_pprint(obj, \u001b[38;5;28mself\u001b[39m, cycle)\n\u001b[1;32m    422\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/IPython/lib/pretty.py:794\u001b[0m, in \u001b[0;36m_repr_pprint\u001b[0;34m(obj, p, cycle)\u001b[0m\n\u001b[1;32m    792\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"A pprint that just redirects to the normal repr function.\"\"\"\u001b[39;00m\n\u001b[1;32m    793\u001b[0m \u001b[38;5;66;03m# Find newlines and replace them with p.break_()\u001b[39;00m\n\u001b[0;32m--> 794\u001b[0m output \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mrepr\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mobj\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    795\u001b[0m lines \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39msplitlines()\n\u001b[1;32m    796\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m p\u001b[38;5;241m.\u001b[39mgroup():\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/nbdev/showdoc.py:168\u001b[0m, in \u001b[0;36mBasicMarkdownRenderer._repr_markdown_\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    166\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21m_repr_markdown_\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[1;32m    167\u001b[0m     doc \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m---\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m'\u001b[39m\n\u001b[0;32m--> 168\u001b[0m     src \u001b[38;5;241m=\u001b[39m \u001b[43mNbdevLookup\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241m.\u001b[39mcode(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mfn)\n\u001b[1;32m    169\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m src: doc \u001b[38;5;241m+\u001b[39m\u001b[38;5;241m=\u001b[39m _ext_link(src, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124msource\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstyle=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mfloat:right; font-size:smaller\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m'\u001b[39m) \u001b[38;5;241m+\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m'\u001b[39m\n\u001b[1;32m    170\u001b[0m     h \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m#\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;241m*\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtitle_level\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/nbdev/doclinks.py:204\u001b[0m, in \u001b[0;36mNbdevLookup.__init__\u001b[0;34m(self, strip_libs, incl_libs, skip_mods)\u001b[0m\n\u001b[1;32m    202\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m incl_libs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m: incl_libs \u001b[38;5;241m=\u001b[39m (L(incl_libs)\u001b[38;5;241m+\u001b[39mstrip_libs)\u001b[38;5;241m.\u001b[39munique()\n\u001b[1;32m    203\u001b[0m \u001b[38;5;66;03m# Dict from lib name to _nbdev module for incl_libs (defaults to all)\u001b[39;00m\n\u001b[0;32m--> 204\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mentries \u001b[38;5;241m=\u001b[39m \u001b[43m{\u001b[49m\u001b[43mo\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mname\u001b[49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43m_qual_syms\u001b[49m\u001b[43m(\u001b[49m\u001b[43mo\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresolve\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mo\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mlist\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mpkg_resources\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43miter_entry_points\u001b[49m\u001b[43m(\u001b[49m\u001b[43mgroup\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mnbdev\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    205\u001b[0m \u001b[43m               \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mincl_libs\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mis\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mo\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdist\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mkey\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mincl_libs\u001b[49m\u001b[43m}\u001b[49m\n\u001b[1;32m    206\u001b[0m py_syms \u001b[38;5;241m=\u001b[39m merge(\u001b[38;5;241m*\u001b[39mL(o[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124msyms\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39mvalues() \u001b[38;5;28;01mfor\u001b[39;00m o \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mentries\u001b[38;5;241m.\u001b[39mvalues())\u001b[38;5;241m.\u001b[39mconcat())\n\u001b[1;32m    207\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m m \u001b[38;5;129;01min\u001b[39;00m strip_libs:\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/nbdev/doclinks.py:204\u001b[0m, in \u001b[0;36m<dictcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m    202\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m incl_libs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m: incl_libs \u001b[38;5;241m=\u001b[39m (L(incl_libs)\u001b[38;5;241m+\u001b[39mstrip_libs)\u001b[38;5;241m.\u001b[39munique()\n\u001b[1;32m    203\u001b[0m \u001b[38;5;66;03m# Dict from lib name to _nbdev module for incl_libs (defaults to all)\u001b[39;00m\n\u001b[0;32m--> 204\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mentries \u001b[38;5;241m=\u001b[39m {o\u001b[38;5;241m.\u001b[39mname: _qual_syms(\u001b[43mo\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresolve\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m) \u001b[38;5;28;01mfor\u001b[39;00m o \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mlist\u001b[39m(pkg_resources\u001b[38;5;241m.\u001b[39miter_entry_points(group\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mnbdev\u001b[39m\u001b[38;5;124m'\u001b[39m))\n\u001b[1;32m    205\u001b[0m                \u001b[38;5;28;01mif\u001b[39;00m incl_libs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mor\u001b[39;00m o\u001b[38;5;241m.\u001b[39mdist\u001b[38;5;241m.\u001b[39mkey \u001b[38;5;129;01min\u001b[39;00m incl_libs}\n\u001b[1;32m    206\u001b[0m py_syms \u001b[38;5;241m=\u001b[39m merge(\u001b[38;5;241m*\u001b[39mL(o[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124msyms\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39mvalues() \u001b[38;5;28;01mfor\u001b[39;00m o \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mentries\u001b[38;5;241m.\u001b[39mvalues())\u001b[38;5;241m.\u001b[39mconcat())\n\u001b[1;32m    207\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m m \u001b[38;5;129;01min\u001b[39;00m strip_libs:\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/pkg_resources/__init__.py:2488\u001b[0m, in \u001b[0;36mEntryPoint.resolve\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   2484\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21mresolve\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[1;32m   2485\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m   2486\u001b[0m \u001b[38;5;124;03m    Resolve the entry point from its module and attrs.\u001b[39;00m\n\u001b[1;32m   2487\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m-> 2488\u001b[0m     module \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43m__import__\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmodule_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfromlist\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m__name__\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlevel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m0\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m   2489\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m   2490\u001b[0m         \u001b[38;5;28;01mreturn\u001b[39;00m functools\u001b[38;5;241m.\u001b[39mreduce(\u001b[38;5;28mgetattr\u001b[39m, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mattrs, module)\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'mlforecast._modidx'"
     ]
    },
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'mlforecast._modidx'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/IPython/core/formatters.py:406\u001b[0m, in \u001b[0;36mBaseFormatter.__call__\u001b[0;34m(self, obj)\u001b[0m\n\u001b[1;32m    404\u001b[0m     method \u001b[38;5;241m=\u001b[39m get_real_method(obj, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mprint_method)\n\u001b[1;32m    405\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m method \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m--> 406\u001b[0m         \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mmethod\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    407\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m    408\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/nbdev/showdoc.py:168\u001b[0m, in \u001b[0;36mBasicMarkdownRenderer._repr_markdown_\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    166\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21m_repr_markdown_\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[1;32m    167\u001b[0m     doc \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m---\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m'\u001b[39m\n\u001b[0;32m--> 168\u001b[0m     src \u001b[38;5;241m=\u001b[39m \u001b[43mNbdevLookup\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241m.\u001b[39mcode(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mfn)\n\u001b[1;32m    169\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m src: doc \u001b[38;5;241m+\u001b[39m\u001b[38;5;241m=\u001b[39m _ext_link(src, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124msource\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstyle=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mfloat:right; font-size:smaller\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m'\u001b[39m) \u001b[38;5;241m+\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m'\u001b[39m\n\u001b[1;32m    170\u001b[0m     h \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m#\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;241m*\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtitle_level\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/nbdev/doclinks.py:204\u001b[0m, in \u001b[0;36mNbdevLookup.__init__\u001b[0;34m(self, strip_libs, incl_libs, skip_mods)\u001b[0m\n\u001b[1;32m    202\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m incl_libs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m: incl_libs \u001b[38;5;241m=\u001b[39m (L(incl_libs)\u001b[38;5;241m+\u001b[39mstrip_libs)\u001b[38;5;241m.\u001b[39munique()\n\u001b[1;32m    203\u001b[0m \u001b[38;5;66;03m# Dict from lib name to _nbdev module for incl_libs (defaults to all)\u001b[39;00m\n\u001b[0;32m--> 204\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mentries \u001b[38;5;241m=\u001b[39m \u001b[43m{\u001b[49m\u001b[43mo\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mname\u001b[49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43m_qual_syms\u001b[49m\u001b[43m(\u001b[49m\u001b[43mo\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresolve\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mo\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mlist\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mpkg_resources\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43miter_entry_points\u001b[49m\u001b[43m(\u001b[49m\u001b[43mgroup\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mnbdev\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    205\u001b[0m \u001b[43m               \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mincl_libs\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mis\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mo\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdist\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mkey\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mincl_libs\u001b[49m\u001b[43m}\u001b[49m\n\u001b[1;32m    206\u001b[0m py_syms \u001b[38;5;241m=\u001b[39m merge(\u001b[38;5;241m*\u001b[39mL(o[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124msyms\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39mvalues() \u001b[38;5;28;01mfor\u001b[39;00m o \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mentries\u001b[38;5;241m.\u001b[39mvalues())\u001b[38;5;241m.\u001b[39mconcat())\n\u001b[1;32m    207\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m m \u001b[38;5;129;01min\u001b[39;00m strip_libs:\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/nbdev/doclinks.py:204\u001b[0m, in \u001b[0;36m<dictcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m    202\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m incl_libs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m: incl_libs \u001b[38;5;241m=\u001b[39m (L(incl_libs)\u001b[38;5;241m+\u001b[39mstrip_libs)\u001b[38;5;241m.\u001b[39munique()\n\u001b[1;32m    203\u001b[0m \u001b[38;5;66;03m# Dict from lib name to _nbdev module for incl_libs (defaults to all)\u001b[39;00m\n\u001b[0;32m--> 204\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mentries \u001b[38;5;241m=\u001b[39m {o\u001b[38;5;241m.\u001b[39mname: _qual_syms(\u001b[43mo\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresolve\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m) \u001b[38;5;28;01mfor\u001b[39;00m o \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mlist\u001b[39m(pkg_resources\u001b[38;5;241m.\u001b[39miter_entry_points(group\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mnbdev\u001b[39m\u001b[38;5;124m'\u001b[39m))\n\u001b[1;32m    205\u001b[0m                \u001b[38;5;28;01mif\u001b[39;00m incl_libs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mor\u001b[39;00m o\u001b[38;5;241m.\u001b[39mdist\u001b[38;5;241m.\u001b[39mkey \u001b[38;5;129;01min\u001b[39;00m incl_libs}\n\u001b[1;32m    206\u001b[0m py_syms \u001b[38;5;241m=\u001b[39m merge(\u001b[38;5;241m*\u001b[39mL(o[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124msyms\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39mvalues() \u001b[38;5;28;01mfor\u001b[39;00m o \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mentries\u001b[38;5;241m.\u001b[39mvalues())\u001b[38;5;241m.\u001b[39mconcat())\n\u001b[1;32m    207\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m m \u001b[38;5;129;01min\u001b[39;00m strip_libs:\n",
      "File \u001b[0;32m~/miniforge3/envs/nixtla/lib/python3.11/site-packages/pkg_resources/__init__.py:2488\u001b[0m, in \u001b[0;36mEntryPoint.resolve\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   2484\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21mresolve\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[1;32m   2485\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m   2486\u001b[0m \u001b[38;5;124;03m    Resolve the entry point from its module and attrs.\u001b[39;00m\n\u001b[1;32m   2487\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m-> 2488\u001b[0m     module \u001b[38;5;241m=\u001b[39m \u001b[38;5;28m__import__\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmodule_name, fromlist\u001b[38;5;241m=\u001b[39m[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m__name__\u001b[39m\u001b[38;5;124m'\u001b[39m], level\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0\u001b[39m)\n\u001b[1;32m   2489\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m   2490\u001b[0m         \u001b[38;5;28;01mreturn\u001b[39;00m functools\u001b[38;5;241m.\u001b[39mreduce(\u001b[38;5;28mgetattr\u001b[39m, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mattrs, module)\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'mlforecast._modidx'"
     ]
    }
   ],
   "source": [
    "#| eval:false\n",
    "show_doc(fill_gaps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to decide how to fill the missing values in the target column. In this tutorial, we will use interpolation, but it is important to consider the specific context of your data when selecting a filling strategy. For example, if you are dealing with daily retail data, a missing value most likely indicates that there were no sales on that day, and you can fill it with zero. Conversely, if you are working with hourly temperature data, a missing value probably means that the sensor was not functioning, and you might prefer to use interpolation to fill the missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "unique_id    0\n",
       "ds           0\n",
       "y            0\n",
       "dtype: int64"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df_complete['y'] = train_df_complete['y'].interpolate(method='linear', limit_direction='both')\n",
    "\n",
    "train_df_complete.isna().sum() # check if there are any missing values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Forecast with TimeGPT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now ready to use the `forecast` method from the `NixtlaClient` class. This method requires the following arguments:\n",
    "\n",
    "- `df`: The DataFrame containing the time series data\n",
    "\n",
    "- `h`: (int) The forecast horizon. In this case, it is 93 days.\n",
    "\n",
    "- `model` (str): The model to use. Default is `timegpt-1`, but since the forecast horizon exceeds the frequency of the data (daily), we will use `timegpt-1-long-horizon`. To learn more about this, please refer to the [Forecasting on a Long Horizon](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: D\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Querying model metadata...\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon, this may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Restricting input...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "fcst = nixtla_client.forecast(train_df_complete, h=len(test_df), model='timegpt-1-long-horizon')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the `plot` method to visualize the `TimeGPT` forecast and the test set. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 122376 characters]",
      "text/plain": [
       "<Figure size 1600x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(test_df, fcst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will use the `evaluate` function from `utilsforecast` to compute the Mean Average Error (MAE) of the TimeGPT forecast. Before proceeding, we need to convert the dates in the forecast to timestamps so we can merge them with the test set.\n",
    "\n",
    "The `evaluate` function requires the following arguments:\n",
    "\n",
    "- `df`: The DataFrame containing the forecast and the actual values (in the `y` column).\n",
    "\n",
    "- `metrics` (list): The metrics to be computed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from utilsforecast.evaluation import evaluate\n",
    "from utilsforecast.losses import mae"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>id1</td>\n",
       "      <td>2022-06-30</td>\n",
       "      <td>13468</td>\n",
       "      <td>13357.357</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>id1</td>\n",
       "      <td>2022-07-01</td>\n",
       "      <td>12932</td>\n",
       "      <td>12390.052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>id1</td>\n",
       "      <td>2022-07-02</td>\n",
       "      <td>9918</td>\n",
       "      <td>9778.649</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>id1</td>\n",
       "      <td>2022-07-03</td>\n",
       "      <td>8967</td>\n",
       "      <td>8846.637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>id1</td>\n",
       "      <td>2022-07-04</td>\n",
       "      <td>12869</td>\n",
       "      <td>11589.071</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id         ds      y    TimeGPT\n",
       "0       id1 2022-06-30  13468  13357.357\n",
       "1       id1 2022-07-01  12932  12390.052\n",
       "2       id1 2022-07-02   9918   9778.649\n",
       "3       id1 2022-07-03   8967   8846.637\n",
       "4       id1 2022-07-04  12869  11589.071"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fcst['ds'] = pd.to_datetime(fcst['ds'])\n",
    "\n",
    "result = test_df.merge(fcst, on=['ds', 'unique_id'], how='left')\n",
    "result.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>metric</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>id1</td>\n",
       "      <td>mae</td>\n",
       "      <td>1824.693059</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id metric      TimeGPT\n",
       "0       id1    mae  1824.693059"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluate(result, metrics=[mae])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Important Considerations "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The key takeaway from this tutorial is that `TimeGPT` requires time series data without missing values. This means that: \n",
    "\n",
    "1. Given the frequency of the data, the timestamps must be continuous, with no gaps between the start and end dates.\n",
    "\n",
    "2. The data must not contain missing values (NaNs). \n",
    "\n",
    "We also showed that `utilsforecast` provides a convenient function to fill missing dates and that you need to decide how to address the missing values. This decision depends on the context of your data, so be mindful when selecting a filling strategy, and choose the one you think best reflects reality.\n",
    "\n",
    "Finally, we also demonstrated that `utilsforecast` provides a function to evaluate the `TimeGPT` forecast using common accuracy metrics. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**References**\n",
    "\n",
    "* [Joaquín Amat Rodrigo and Javier Escobar Ortiz (2022). \"Exclude covid impact in time series forecasting\"](https://www.cienciadedatos.net/documentos/py45-weighted-time-series-forecasting.html)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
