{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "695cebbe",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "!pip install -Uqq nixtla datasetsforecast"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "579b1471",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "from nixtla.utils import in_colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "93693704",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "IN_COLAB = in_colab()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29ff4759",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    from nixtla.utils import colab_badge\n",
    "    from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24c899f2-78c1-43b2-8347-3164e3549c3f",
   "metadata": {},
   "source": [
    "# Categorical variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a81fc39a-c6a0-485d-a3f3-c3a6298928a6",
   "metadata": {},
   "source": [
    "Categorical variables are external factors that can influence a forecast. These variables take on one of a limited, fixed number of possible values, and induce a grouping of your observations.\\\n",
    "\\\n",
    "For example, if you're forecasting daily product demand for a retailer, you could benefit from an event variable that may tell you what kind of event takes place on a given day, for example 'None', 'Sporting', or 'Cultural'.\\\n",
    "\\\n",
    "To incorporate categorical variables in TimeGPT, you'll need to pair each point in your time series data with the corresponding external data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bc1e229-6c80-463f-b721-465a48fbddf1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/tutorials/03_categorical_variables.ipynb)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#| echo: false\n",
    "if not IN_COLAB:\n",
    "    load_dotenv()    \n",
    "    colab_badge('docs/tutorials/03_categorical_variables')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "957acb84",
   "metadata": {},
   "source": [
    "## 1. Import packages\n",
    "First, we install and import the required packages and initialize the Nixtla client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a84a0f65-e084-4e65-a0fb-d27c184dde44",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import os\n",
    "\n",
    "from nixtla import NixtlaClient\n",
    "from datasetsforecast.m5 import M5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "469d474a-c427-427c-a127-d140aeba0354",
   "metadata": {},
   "outputs": [],
   "source": [
    "nixtla_client = NixtlaClient(\n",
    "    # defaults to os.environ.get(\"NIXTLA_API_KEY\")\n",
    "    api_key = 'my_api_key_provided_by_nixtla'   \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e38badfe",
   "metadata": {},
   "source": [
    "> 👍 Use an Azure AI endpoint\n",
    "> \n",
    "> To use an Azure AI endpoint, remember to set also the `base_url` argument:\n",
    "> \n",
    "> `nixtla_client = NixtlaClient(base_url=\"you azure ai endpoint\", api_key=\"your api_key\")`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ced2bba4-089d-4733-9613-33a16cad622b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    nixtla_client = NixtlaClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b6f16ac",
   "metadata": {},
   "source": [
    "## 2. Load M5 data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc2bb3db-00e6-44e6-8dc3-a2e0eba7e295",
   "metadata": {},
   "source": [
    "Let's see an example on predicting sales of products of the [M5 dataset](https://nixtlaverse.nixtla.io/datasetsforecast/m5.html). The M5 dataset contains daily product demand (sales) for 10 retail stores in the US.\\\n",
    "\\\n",
    "First, we load the data using `datasetsforecast`. This returns:\n",
    "\n",
    "* `Y_df`, containing the sales (`y` column), for each unique product (`unique_id` column) at every timestamp (`ds` column). \n",
    "* `X_df`, containing additional relevant information for each unique product (`unique_id` column) at every timestamp (`ds` column). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fec19dc-48dd-4337-8678-fe3753b5eb30",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-01-29</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-01-30</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-01-31</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-01</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-02</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-03</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-04</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-05</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-06</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-07</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          unique_id         ds    y\n",
       "0  FOODS_1_001_CA_1 2011-01-29  3.0\n",
       "1  FOODS_1_001_CA_1 2011-01-30  0.0\n",
       "2  FOODS_1_001_CA_1 2011-01-31  0.0\n",
       "3  FOODS_1_001_CA_1 2011-02-01  1.0\n",
       "4  FOODS_1_001_CA_1 2011-02-02  4.0\n",
       "5  FOODS_1_001_CA_1 2011-02-03  2.0\n",
       "6  FOODS_1_001_CA_1 2011-02-04  0.0\n",
       "7  FOODS_1_001_CA_1 2011-02-05  2.0\n",
       "8  FOODS_1_001_CA_1 2011-02-06  0.0\n",
       "9  FOODS_1_001_CA_1 2011-02-07  0.0"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#| eval: false\n",
    "Y_df, X_df, _ = M5.load(directory=os.getcwd())\n",
    "Y_df['ds'] = pd.to_datetime(Y_df['ds'])\n",
    "X_df['ds'] = pd.to_datetime(X_df['ds'])\n",
    "Y_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ed2196b",
   "metadata": {},
   "source": [
    "For this example, we will only keep the additional relevant information from the column `event_type_1`. This column is a _categorical variable_ that indicates whether an important event that might affect the sales of the product takes place at a certain date. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "105dfb00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>event_type_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-01-29</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-01-30</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-01-31</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-01</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-02</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-03</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-04</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-05</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-06</td>\n",
       "      <td>Sporting</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>FOODS_1_001_CA_1</td>\n",
       "      <td>2011-02-07</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          unique_id         ds event_type_1\n",
       "0  FOODS_1_001_CA_1 2011-01-29          nan\n",
       "1  FOODS_1_001_CA_1 2011-01-30          nan\n",
       "2  FOODS_1_001_CA_1 2011-01-31          nan\n",
       "3  FOODS_1_001_CA_1 2011-02-01          nan\n",
       "4  FOODS_1_001_CA_1 2011-02-02          nan\n",
       "5  FOODS_1_001_CA_1 2011-02-03          nan\n",
       "6  FOODS_1_001_CA_1 2011-02-04          nan\n",
       "7  FOODS_1_001_CA_1 2011-02-05          nan\n",
       "8  FOODS_1_001_CA_1 2011-02-06     Sporting\n",
       "9  FOODS_1_001_CA_1 2011-02-07          nan"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#| eval: false\n",
    "X_df = X_df[['unique_id', 'ds', 'event_type_1']]\n",
    "\n",
    "X_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dbc97b8",
   "metadata": {},
   "source": [
    "As you can see, on February 6th 2011, there is a Sporting event. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b360eb3d",
   "metadata": {},
   "source": [
    "## 3. Forecasting product demand using categorical variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2d60ea6",
   "metadata": {},
   "source": [
    "We will forecast the demand for a single product only. We choose a high selling food product identified by `FOODS_3_090_CA_3`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29fe0c38",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| eval: false\n",
    "product = 'FOODS_3_090_CA_3'\n",
    "Y_df_product = Y_df.query('unique_id == @product')\n",
    "X_df_product = X_df.query('unique_id == @product')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3f5a78f",
   "metadata": {},
   "source": [
    "We merge our two dataframes to create the dataset to be used in TimeGPT."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c4d8b75",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>event_type_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-01-29</td>\n",
       "      <td>108.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-01-30</td>\n",
       "      <td>132.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-01-31</td>\n",
       "      <td>102.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-02-01</td>\n",
       "      <td>120.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-02-02</td>\n",
       "      <td>106.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-02-03</td>\n",
       "      <td>123.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-02-04</td>\n",
       "      <td>279.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-02-05</td>\n",
       "      <td>175.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-02-06</td>\n",
       "      <td>186.0</td>\n",
       "      <td>Sporting</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2011-02-07</td>\n",
       "      <td>120.0</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          unique_id         ds      y event_type_1\n",
       "0  FOODS_3_090_CA_3 2011-01-29  108.0          nan\n",
       "1  FOODS_3_090_CA_3 2011-01-30  132.0          nan\n",
       "2  FOODS_3_090_CA_3 2011-01-31  102.0          nan\n",
       "3  FOODS_3_090_CA_3 2011-02-01  120.0          nan\n",
       "4  FOODS_3_090_CA_3 2011-02-02  106.0          nan\n",
       "5  FOODS_3_090_CA_3 2011-02-03  123.0          nan\n",
       "6  FOODS_3_090_CA_3 2011-02-04  279.0          nan\n",
       "7  FOODS_3_090_CA_3 2011-02-05  175.0          nan\n",
       "8  FOODS_3_090_CA_3 2011-02-06  186.0     Sporting\n",
       "9  FOODS_3_090_CA_3 2011-02-07  120.0          nan"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#| eval: false\n",
    "df = Y_df_product.merge(X_df_product)\n",
    "\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8171581",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    df = pd.read_parquet(\"../../assets/M5_categorical_variables_example.parquet\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47873237",
   "metadata": {},
   "source": [
    "In order to use _categorical variables_ with TimeGPT, it is necessary to numerically encode the variables. We will use _one-hot encoding_ in this tutorial. \n",
    "\n",
    "We can one-hot encode the `event_type_1` column by using pandas built-in `get_dummies` functionality. After one-hot encoding the `event_type_1` variable, we can add it to the dataframe and remove the original column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee09d3c0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>Cultural</th>\n",
       "      <th>National</th>\n",
       "      <th>Religious</th>\n",
       "      <th>Sporting</th>\n",
       "      <th>nan</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1959</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-10</td>\n",
       "      <td>140.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1960</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-11</td>\n",
       "      <td>151.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1961</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-12</td>\n",
       "      <td>87.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1962</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-13</td>\n",
       "      <td>67.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1963</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-14</td>\n",
       "      <td>50.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1964</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-15</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1965</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-16</td>\n",
       "      <td>116.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1966</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-17</td>\n",
       "      <td>124.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1967</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-18</td>\n",
       "      <td>167.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1968</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-06-19</td>\n",
       "      <td>118.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             unique_id         ds      y  Cultural  National  Religious  \\\n",
       "1959  FOODS_3_090_CA_3 2016-06-10  140.0         0         0          0   \n",
       "1960  FOODS_3_090_CA_3 2016-06-11  151.0         0         0          0   \n",
       "1961  FOODS_3_090_CA_3 2016-06-12   87.0         0         0          0   \n",
       "1962  FOODS_3_090_CA_3 2016-06-13   67.0         0         0          0   \n",
       "1963  FOODS_3_090_CA_3 2016-06-14   50.0         0         0          0   \n",
       "1964  FOODS_3_090_CA_3 2016-06-15   58.0         0         0          0   \n",
       "1965  FOODS_3_090_CA_3 2016-06-16  116.0         0         0          0   \n",
       "1966  FOODS_3_090_CA_3 2016-06-17  124.0         0         0          0   \n",
       "1967  FOODS_3_090_CA_3 2016-06-18  167.0         0         0          0   \n",
       "1968  FOODS_3_090_CA_3 2016-06-19  118.0         0         0          0   \n",
       "\n",
       "      Sporting  nan  \n",
       "1959         0    1  \n",
       "1960         0    1  \n",
       "1961         0    1  \n",
       "1962         0    1  \n",
       "1963         0    1  \n",
       "1964         0    1  \n",
       "1965         0    1  \n",
       "1966         0    1  \n",
       "1967         0    1  \n",
       "1968         1    0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "event_type_1_ohe = pd.get_dummies(df['event_type_1'], dtype=int)\n",
    "df = pd.concat([df, event_type_1_ohe], axis=1)\n",
    "df = df.drop(columns = 'event_type_1')\n",
    "\n",
    "df.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c86f2a4",
   "metadata": {},
   "source": [
    "As you can see, we have now added 5 columns, each with a binary indicator (`1` or `0`) whether there is a `Cultural`, `National`, `Religious`, `Sporting` or no (`nan`) event on that particular day. For example, on June 19th 2016, there is a `Sporting` event."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ef0388c",
   "metadata": {},
   "source": [
    "Let's turn to our forecasting task. We will forecast the first 7 days of February 2016. This includes 7 February 2016 - the date on which [Super Bowl 50](https://en.wikipedia.org/wiki/Super_Bowl_50) was held. Such large, national events typically impact retail product sales.\n",
    "\n",
    "To use the encoded categorical variables in TimeGPT, we have to add them as future values. Therefore, we create a future values dataframe, that contains the `unique_id`, the timestamp `ds`, and the encoded categorical variables.\n",
    "\n",
    "Of course, we drop the target column as this is normally not available - this is the quantity that we seek to forecast!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c17c641",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>Cultural</th>\n",
       "      <th>National</th>\n",
       "      <th>Religious</th>\n",
       "      <th>Sporting</th>\n",
       "      <th>nan</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1829</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-01</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1830</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-02</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1831</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-03</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1832</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-04</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1833</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-05</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1834</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-06</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1835</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-07</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             unique_id         ds  Cultural  National  Religious  Sporting  \\\n",
       "1829  FOODS_3_090_CA_3 2016-02-01         0         0          0         0   \n",
       "1830  FOODS_3_090_CA_3 2016-02-02         0         0          0         0   \n",
       "1831  FOODS_3_090_CA_3 2016-02-03         0         0          0         0   \n",
       "1832  FOODS_3_090_CA_3 2016-02-04         0         0          0         0   \n",
       "1833  FOODS_3_090_CA_3 2016-02-05         0         0          0         0   \n",
       "1834  FOODS_3_090_CA_3 2016-02-06         0         0          0         0   \n",
       "1835  FOODS_3_090_CA_3 2016-02-07         0         0          0         1   \n",
       "\n",
       "      nan  \n",
       "1829    1  \n",
       "1830    1  \n",
       "1831    1  \n",
       "1832    1  \n",
       "1833    1  \n",
       "1834    1  \n",
       "1835    0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "future_ex_vars_df = df.drop(columns = ['y'])\n",
    "future_ex_vars_df = future_ex_vars_df.query(\"ds >= '2016-02-01' & ds <= '2016-02-07'\")\n",
    "\n",
    "future_ex_vars_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09663187",
   "metadata": {},
   "source": [
    "Next, we limit our input dataframe to all but the 7 forecast days:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9955cc75",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>Cultural</th>\n",
       "      <th>National</th>\n",
       "      <th>Religious</th>\n",
       "      <th>Sporting</th>\n",
       "      <th>nan</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1819</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-22</td>\n",
       "      <td>94.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1820</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-23</td>\n",
       "      <td>144.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1821</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-24</td>\n",
       "      <td>146.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1822</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-25</td>\n",
       "      <td>87.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1823</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-26</td>\n",
       "      <td>73.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1824</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-27</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1825</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-28</td>\n",
       "      <td>64.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1826</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-29</td>\n",
       "      <td>102.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1827</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-30</td>\n",
       "      <td>113.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1828</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-01-31</td>\n",
       "      <td>98.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             unique_id         ds      y  Cultural  National  Religious  \\\n",
       "1819  FOODS_3_090_CA_3 2016-01-22   94.0         0         0          0   \n",
       "1820  FOODS_3_090_CA_3 2016-01-23  144.0         0         0          0   \n",
       "1821  FOODS_3_090_CA_3 2016-01-24  146.0         0         0          0   \n",
       "1822  FOODS_3_090_CA_3 2016-01-25   87.0         0         0          0   \n",
       "1823  FOODS_3_090_CA_3 2016-01-26   73.0         0         0          0   \n",
       "1824  FOODS_3_090_CA_3 2016-01-27   62.0         0         0          0   \n",
       "1825  FOODS_3_090_CA_3 2016-01-28   64.0         0         0          0   \n",
       "1826  FOODS_3_090_CA_3 2016-01-29  102.0         0         0          0   \n",
       "1827  FOODS_3_090_CA_3 2016-01-30  113.0         0         0          0   \n",
       "1828  FOODS_3_090_CA_3 2016-01-31   98.0         0         0          0   \n",
       "\n",
       "      Sporting  nan  \n",
       "1819         0    1  \n",
       "1820         0    1  \n",
       "1821         0    1  \n",
       "1822         0    1  \n",
       "1823         0    1  \n",
       "1824         0    1  \n",
       "1825         0    1  \n",
       "1826         0    1  \n",
       "1827         0    1  \n",
       "1828         0    1  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train = df.query(\"ds < '2016-02-01'\")\n",
    "\n",
    "df_train.tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99f1e41d-e5bf-4d01-aa68-1a7a7fbb579b",
   "metadata": {},
   "source": [
    "Let's call the `forecast` method, first _without_ the categorical variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d252a0e0-f393-4957-8173-230972fc7a40",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: D\n",
      "INFO:nixtla.nixtla_client:Restricting input...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>TimeGPT-lo-90</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "      <th>TimeGPT-hi-90</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-01</td>\n",
       "      <td>73.304092</td>\n",
       "      <td>53.449049</td>\n",
       "      <td>54.795078</td>\n",
       "      <td>91.813107</td>\n",
       "      <td>93.159136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-02</td>\n",
       "      <td>66.335518</td>\n",
       "      <td>47.510669</td>\n",
       "      <td>50.274136</td>\n",
       "      <td>82.396899</td>\n",
       "      <td>85.160367</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-03</td>\n",
       "      <td>65.881630</td>\n",
       "      <td>36.218617</td>\n",
       "      <td>41.388896</td>\n",
       "      <td>90.374364</td>\n",
       "      <td>95.544643</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-04</td>\n",
       "      <td>72.371864</td>\n",
       "      <td>-26.683115</td>\n",
       "      <td>25.097362</td>\n",
       "      <td>119.646367</td>\n",
       "      <td>171.426844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-05</td>\n",
       "      <td>95.141045</td>\n",
       "      <td>-2.084882</td>\n",
       "      <td>34.027078</td>\n",
       "      <td>156.255011</td>\n",
       "      <td>192.366971</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          unique_id          ds    TimeGPT  TimeGPT-lo-90  TimeGPT-lo-80  \\\n",
       "0  FOODS_3_090_CA_3  2016-02-01  73.304092      53.449049      54.795078   \n",
       "1  FOODS_3_090_CA_3  2016-02-02  66.335518      47.510669      50.274136   \n",
       "2  FOODS_3_090_CA_3  2016-02-03  65.881630      36.218617      41.388896   \n",
       "3  FOODS_3_090_CA_3  2016-02-04  72.371864     -26.683115      25.097362   \n",
       "4  FOODS_3_090_CA_3  2016-02-05  95.141045      -2.084882      34.027078   \n",
       "\n",
       "   TimeGPT-hi-80  TimeGPT-hi-90  \n",
       "0      91.813107      93.159136  \n",
       "1      82.396899      85.160367  \n",
       "2      90.374364      95.544643  \n",
       "3     119.646367     171.426844  \n",
       "4     156.255011     192.366971  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timegpt_fcst_without_cat_vars_df = nixtla_client.forecast(df=df_train, h=7, level=[80, 90])\n",
    "timegpt_fcst_without_cat_vars_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99d2d153",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c33a6a5",
   "metadata": {},
   "source": [
    "We plot the forecast and the last 28 days before the forecast period:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18f36e5c-f41f-4888-b279-97558b71c1bf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 83848 characters]",
      "text/plain": [
       "<Figure size 2400x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df[['unique_id', 'ds', 'y']].query(\"ds <= '2016-02-07'\"), \n",
    "    timegpt_fcst_without_cat_vars_df, \n",
    "    max_insample_length=28, \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6028d3f",
   "metadata": {},
   "source": [
    "TimeGPT already provides a reasonable forecast, but it seems to somewhat underforecast the peak on the 6th of February 2016 - the day before the Super Bowl."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bede86e",
   "metadata": {},
   "source": [
    "Let's call the `forecast` method again, now _with_ the categorical variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "928502f3",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: D\n",
      "INFO:nixtla.nixtla_client:Using the following exogenous variables: Cultural, National, Religious, Sporting, nan\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>TimeGPT-lo-90</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "      <th>TimeGPT-hi-90</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-01</td>\n",
       "      <td>70.661271</td>\n",
       "      <td>-0.204378</td>\n",
       "      <td>14.593348</td>\n",
       "      <td>126.729194</td>\n",
       "      <td>141.526919</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-02</td>\n",
       "      <td>65.566941</td>\n",
       "      <td>-20.394326</td>\n",
       "      <td>11.654239</td>\n",
       "      <td>119.479643</td>\n",
       "      <td>151.528208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-03</td>\n",
       "      <td>68.510010</td>\n",
       "      <td>-33.713710</td>\n",
       "      <td>6.732952</td>\n",
       "      <td>130.287069</td>\n",
       "      <td>170.733731</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-04</td>\n",
       "      <td>75.417710</td>\n",
       "      <td>-40.974649</td>\n",
       "      <td>4.751767</td>\n",
       "      <td>146.083653</td>\n",
       "      <td>191.810069</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>2016-02-05</td>\n",
       "      <td>97.340302</td>\n",
       "      <td>-57.385361</td>\n",
       "      <td>18.253812</td>\n",
       "      <td>176.426792</td>\n",
       "      <td>252.065965</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          unique_id          ds    TimeGPT  TimeGPT-lo-90  TimeGPT-lo-80  \\\n",
       "0  FOODS_3_090_CA_3  2016-02-01  70.661271      -0.204378      14.593348   \n",
       "1  FOODS_3_090_CA_3  2016-02-02  65.566941     -20.394326      11.654239   \n",
       "2  FOODS_3_090_CA_3  2016-02-03  68.510010     -33.713710       6.732952   \n",
       "3  FOODS_3_090_CA_3  2016-02-04  75.417710     -40.974649       4.751767   \n",
       "4  FOODS_3_090_CA_3  2016-02-05  97.340302     -57.385361      18.253812   \n",
       "\n",
       "   TimeGPT-hi-80  TimeGPT-hi-90  \n",
       "0     126.729194     141.526919  \n",
       "1     119.479643     151.528208  \n",
       "2     130.287069     170.733731  \n",
       "3     146.083653     191.810069  \n",
       "4     176.426792     252.065965  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timegpt_fcst_with_cat_vars_df = nixtla_client.forecast(df=df_train, X_df=future_ex_vars_df, h=7, level=[80, 90])\n",
    "timegpt_fcst_with_cat_vars_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67812465",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82686c05",
   "metadata": {},
   "source": [
    "We plot the forecast and the last 28 days before the forecast period:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fd659c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 82740 characters]",
      "text/plain": [
       "<Figure size 2400x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df[['unique_id', 'ds', 'y']].query(\"ds <= '2016-02-07'\"), \n",
    "    timegpt_fcst_with_cat_vars_df, \n",
    "    max_insample_length=28, \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb892aba",
   "metadata": {},
   "source": [
    "We can visually verify that the forecast is closer to the actual observed value, which is the result of including the categorical variable in our forecast."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93499305",
   "metadata": {},
   "source": [
    "Let's verify this conclusion by computing the [Mean Absolute Error](https://en.wikipedia.org/wiki/Mean_absolute_error) on the forecasts we created. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65cdaab0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from utilsforecast.losses import mae"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "de1fd15f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create target dataframe\n",
    "df_target = df[['unique_id', 'ds', 'y']].query(\"ds >= '2016-02-01' & ds <= '2016-02-07'\")\n",
    "\n",
    "# Rename forecast columns\n",
    "timegpt_fcst_without_cat_vars_df = timegpt_fcst_without_cat_vars_df.rename(columns={'TimeGPT': 'TimeGPT-without-cat-vars'})\n",
    "timegpt_fcst_with_cat_vars_df = timegpt_fcst_with_cat_vars_df.rename(columns={'TimeGPT': 'TimeGPT-with-cat-vars'})\n",
    "\n",
    "# Merge forecasts with target dataframe\n",
    "df_target = df_target.merge(timegpt_fcst_without_cat_vars_df[['unique_id', 'ds', 'TimeGPT-without-cat-vars']])\n",
    "df_target = df_target.merge(timegpt_fcst_with_cat_vars_df[['unique_id', 'ds', 'TimeGPT-with-cat-vars']])\n",
    "\n",
    "# Compute errors\n",
    "mean_absolute_errors = mae(df_target, ['TimeGPT-without-cat-vars', 'TimeGPT-with-cat-vars'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8c87654",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>TimeGPT-without-cat-vars</th>\n",
       "      <th>TimeGPT-with-cat-vars</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FOODS_3_090_CA_3</td>\n",
       "      <td>24.285649</td>\n",
       "      <td>20.028514</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          unique_id  TimeGPT-without-cat-vars  TimeGPT-with-cat-vars\n",
       "0  FOODS_3_090_CA_3                 24.285649              20.028514"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_absolute_errors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5938cd01",
   "metadata": {},
   "source": [
    "Indeed, we find that the error when using TimeGPT with the categorical variable is approx. 20% lower than when using TimeGPT without the categorical variables, indicating better performance when we include the categorical variable."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
