{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0edb5df1",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "!pip install -Uqq nixtla"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81059425",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "from nixtla.utils import in_colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b97694a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "IN_COLAB = in_colab()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44ec954b-63b3-4236-86a5-5a2cd86de934",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    from nixtla.utils import colab_badge\n",
    "    from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24c899f2-78c1-43b2-8347-3164e3549c3f",
   "metadata": {},
   "source": [
    "# Exogenous variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a81fc39a-c6a0-485d-a3f3-c3a6298928a6",
   "metadata": {},
   "source": [
    "Exogenous variables or external factors are crucial in time series forecasting as they provide additional information that might influence the prediction. These variables could include holiday markers, marketing spending, weather data, or any other external data that correlate with the time series data you are forecasting.\n",
    "\n",
    "For example, if you're forecasting ice cream sales, temperature data could serve as a useful exogenous variable. On hotter days, ice cream sales may increase.\n",
    "\n",
    "To incorporate exogenous variables in TimeGPT, you'll need to pair each point in your time series data with the corresponding external data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bc1e229-6c80-463f-b721-465a48fbddf1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/tutorials/01_exogenous_variables.ipynb)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#| echo: false\n",
    "if not IN_COLAB:\n",
    "    load_dotenv()\n",
    "    colab_badge('docs/tutorials/01_exogenous_variables')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfa2ede9",
   "metadata": {},
   "source": [
    "## 1. Import packages\n",
    "First, we import the required packages and initialize the Nixtla client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a84a0f65-e084-4e65-a0fb-d27c184dde44",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from nixtla import NixtlaClient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "469d474a-c427-427c-a127-d140aeba0354",
   "metadata": {},
   "outputs": [],
   "source": [
    "nixtla_client = NixtlaClient(\n",
    "    # defaults to os.environ.get(\"NIXTLA_API_KEY\")\n",
    "    api_key = 'my_api_key_provided_by_nixtla'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de88de33",
   "metadata": {},
   "source": [
    "> 👍 Use an Azure AI endpoint\n",
    "> \n",
    "> To use an Azure AI endpoint, remember to set also the `base_url` argument:\n",
    "> \n",
    "> `nixtla_client = NixtlaClient(base_url=\"you azure ai endpoint\", api_key=\"your api_key\")`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ced2bba4-089d-4733-9613-33a16cad622b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    nixtla_client = NixtlaClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "054c3cad",
   "metadata": {},
   "source": [
    "## 2. Load data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc2bb3db-00e6-44e6-8dc3-a2e0eba7e295",
   "metadata": {},
   "source": [
    "Let's see an example on predicting day-ahead electricity prices. The following dataset contains the hourly electricity price (`y` column) for five markets in Europe and US, identified by the `unique_id` column. The columns from `Exogenous1` to `day_6` are exogenous variables that TimeGPT will use to predict the prices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fec19dc-48dd-4337-8678-fe3753b5eb30",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>Exogenous1</th>\n",
       "      <th>Exogenous2</th>\n",
       "      <th>day_0</th>\n",
       "      <th>day_1</th>\n",
       "      <th>day_2</th>\n",
       "      <th>day_3</th>\n",
       "      <th>day_4</th>\n",
       "      <th>day_5</th>\n",
       "      <th>day_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 00:00:00</td>\n",
       "      <td>70.00</td>\n",
       "      <td>57253.0</td>\n",
       "      <td>49593.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 01:00:00</td>\n",
       "      <td>37.10</td>\n",
       "      <td>51887.0</td>\n",
       "      <td>46073.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 02:00:00</td>\n",
       "      <td>37.10</td>\n",
       "      <td>51896.0</td>\n",
       "      <td>44927.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 03:00:00</td>\n",
       "      <td>44.75</td>\n",
       "      <td>48428.0</td>\n",
       "      <td>44483.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 04:00:00</td>\n",
       "      <td>37.10</td>\n",
       "      <td>46721.0</td>\n",
       "      <td>44338.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                   ds      y  Exogenous1  Exogenous2  day_0  day_1  \\\n",
       "0        BE  2016-10-22 00:00:00  70.00     57253.0     49593.0    0.0    0.0   \n",
       "1        BE  2016-10-22 01:00:00  37.10     51887.0     46073.0    0.0    0.0   \n",
       "2        BE  2016-10-22 02:00:00  37.10     51896.0     44927.0    0.0    0.0   \n",
       "3        BE  2016-10-22 03:00:00  44.75     48428.0     44483.0    0.0    0.0   \n",
       "4        BE  2016-10-22 04:00:00  37.10     46721.0     44338.0    0.0    0.0   \n",
       "\n",
       "   day_2  day_3  day_4  day_5  day_6  \n",
       "0    0.0    0.0    0.0    1.0    0.0  \n",
       "1    0.0    0.0    0.0    1.0    0.0  \n",
       "2    0.0    0.0    0.0    1.0    0.0  \n",
       "3    0.0    0.0    0.0    1.0    0.0  \n",
       "4    0.0    0.0    0.0    1.0    0.0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short-with-ex-vars.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8f00038",
   "metadata": {},
   "source": [
    "## 3a. Forecasting electricity prices using future exogenous variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eed41a2f-67ce-4812-8073-18b271b1592d",
   "metadata": {},
   "source": [
    "To produce forecasts with future exogenous variables we have to add the future values of the exogenous variables. Let's read this dataset. In this case, we want to predict 24 steps ahead, therefore each `unique_id` will have 24 observations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95c03577-25f3-479f-a76e-fd5e4632da96",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>Exogenous1</th>\n",
       "      <th>Exogenous2</th>\n",
       "      <th>day_0</th>\n",
       "      <th>day_1</th>\n",
       "      <th>day_2</th>\n",
       "      <th>day_3</th>\n",
       "      <th>day_4</th>\n",
       "      <th>day_5</th>\n",
       "      <th>day_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>70318.0</td>\n",
       "      <td>64108.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>67898.0</td>\n",
       "      <td>62492.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>68379.0</td>\n",
       "      <td>61571.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>64972.0</td>\n",
       "      <td>60381.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>62900.0</td>\n",
       "      <td>60298.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                   ds  Exogenous1  Exogenous2  day_0  day_1  day_2  \\\n",
       "0        BE  2016-12-31 00:00:00     70318.0     64108.0    0.0    0.0    0.0   \n",
       "1        BE  2016-12-31 01:00:00     67898.0     62492.0    0.0    0.0    0.0   \n",
       "2        BE  2016-12-31 02:00:00     68379.0     61571.0    0.0    0.0    0.0   \n",
       "3        BE  2016-12-31 03:00:00     64972.0     60381.0    0.0    0.0    0.0   \n",
       "4        BE  2016-12-31 04:00:00     62900.0     60298.0    0.0    0.0    0.0   \n",
       "\n",
       "   day_3  day_4  day_5  day_6  \n",
       "0    0.0    0.0    1.0    0.0  \n",
       "1    0.0    0.0    1.0    0.0  \n",
       "2    0.0    0.0    1.0    0.0  \n",
       "3    0.0    0.0    1.0    0.0  \n",
       "4    0.0    0.0    1.0    0.0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "future_ex_vars_df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short-future-ex-vars.csv')\n",
    "future_ex_vars_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99f1e41d-e5bf-4d01-aa68-1a7a7fbb579b",
   "metadata": {},
   "source": [
    "Let's call the `forecast` method, adding this information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d252a0e0-f393-4957-8173-230972fc7a40",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Querying model metadata...\n",
      "INFO:nixtla.nixtla_client:Using future exogenous features: ['Exogenous1', 'Exogenous2', 'day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6']\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "      <th>TimeGPT-hi-90</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-lo-90</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>51.632830</td>\n",
       "      <td>61.598820</td>\n",
       "      <td>66.088295</td>\n",
       "      <td>41.666843</td>\n",
       "      <td>37.177372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>45.750877</td>\n",
       "      <td>54.611988</td>\n",
       "      <td>60.176445</td>\n",
       "      <td>36.889767</td>\n",
       "      <td>31.325312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>39.650543</td>\n",
       "      <td>46.256210</td>\n",
       "      <td>52.842808</td>\n",
       "      <td>33.044876</td>\n",
       "      <td>26.458277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>34.000072</td>\n",
       "      <td>44.015310</td>\n",
       "      <td>47.429000</td>\n",
       "      <td>23.984835</td>\n",
       "      <td>20.571144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>33.785370</td>\n",
       "      <td>43.140503</td>\n",
       "      <td>48.581240</td>\n",
       "      <td>24.430239</td>\n",
       "      <td>18.989498</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                  ds    TimeGPT  TimeGPT-hi-80  TimeGPT-hi-90  \\\n",
       "0        BE 2016-12-31 00:00:00  51.632830      61.598820      66.088295   \n",
       "1        BE 2016-12-31 01:00:00  45.750877      54.611988      60.176445   \n",
       "2        BE 2016-12-31 02:00:00  39.650543      46.256210      52.842808   \n",
       "3        BE 2016-12-31 03:00:00  34.000072      44.015310      47.429000   \n",
       "4        BE 2016-12-31 04:00:00  33.785370      43.140503      48.581240   \n",
       "\n",
       "   TimeGPT-lo-80  TimeGPT-lo-90  \n",
       "0      41.666843      37.177372  \n",
       "1      36.889767      31.325312  \n",
       "2      33.044876      26.458277  \n",
       "3      23.984835      20.571144  \n",
       "4      24.430239      18.989498  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timegpt_fcst_ex_vars_df = nixtla_client.forecast(df=df, X_df=future_ex_vars_df, h=24, level=[80, 90])\n",
    "timegpt_fcst_ex_vars_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12207fa7",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18f36e5c-f41f-4888-b279-97558b71c1bf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 232760 characters]",
      "text/plain": [
       "<Figure size 1600x700 with 4 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df[['unique_id', 'ds', 'y']], \n",
    "    timegpt_fcst_ex_vars_df, \n",
    "    max_insample_length=365, \n",
    "    level=[80, 90], \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e945ad3c-63fd-4e51-9815-336306f60463",
   "metadata": {},
   "source": [
    "We can also show the importance of the features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef1c9df3-eff2-4984-a88f-00274b21b3cd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: ylabel='features'>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 25320 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nixtla_client.weights_x.plot.barh(x='features', y='weights')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80c432bd",
   "metadata": {},
   "source": [
    "This plot shows that `Exogenous1` and `Exogenous2` are the most important for this forecasting task, as they have the largest weight."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6476dd08",
   "metadata": {},
   "source": [
    "## 3b. Forecasting electricity prices using historic exogenous variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "812064d9",
   "metadata": {},
   "source": [
    "In the example above, we just loaded the future exogenous variables. Often, these are not available because these variables are unknown. We can also make forecasts using only historic exogenous variables. This can be done by adding the `hist_exog_list` argument with the list of columns of `df` to be considered as historical. In that case, we can pass all extra columns available in `df` as historic exogenous variables using `hist_exog_list=['Exogenous1', 'Exogenous2', 'day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6']`.\n",
    "\n",
    "::: {.callout-important}\n",
    "If you include historic exogenous variables in your model, you are _implicitly_ making assumptions about the future of these exogenous variables in your forecast. It is recommended to make these assumptions explicit by making use of future exogenous variables.\n",
    "::: "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12123021",
   "metadata": {},
   "source": [
    "Let's call the `forecast` method, adding `hist_exog_list`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "671bea25",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Using historical exogenous features: ['Exogenous1', 'Exogenous2', 'day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6']\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "      <th>TimeGPT-hi-90</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-lo-90</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>47.311330</td>\n",
       "      <td>57.277317</td>\n",
       "      <td>61.766790</td>\n",
       "      <td>37.345340</td>\n",
       "      <td>32.855870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>47.142740</td>\n",
       "      <td>56.003850</td>\n",
       "      <td>61.568306</td>\n",
       "      <td>38.281628</td>\n",
       "      <td>32.717170</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>47.311474</td>\n",
       "      <td>53.917137</td>\n",
       "      <td>60.503740</td>\n",
       "      <td>40.705810</td>\n",
       "      <td>34.119210</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>47.224514</td>\n",
       "      <td>57.239750</td>\n",
       "      <td>60.653442</td>\n",
       "      <td>37.209280</td>\n",
       "      <td>33.795586</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>47.266945</td>\n",
       "      <td>56.622078</td>\n",
       "      <td>62.062817</td>\n",
       "      <td>37.911810</td>\n",
       "      <td>32.471073</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                  ds    TimeGPT  TimeGPT-hi-80  TimeGPT-hi-90  \\\n",
       "0        BE 2016-12-31 00:00:00  47.311330      57.277317      61.766790   \n",
       "1        BE 2016-12-31 01:00:00  47.142740      56.003850      61.568306   \n",
       "2        BE 2016-12-31 02:00:00  47.311474      53.917137      60.503740   \n",
       "3        BE 2016-12-31 03:00:00  47.224514      57.239750      60.653442   \n",
       "4        BE 2016-12-31 04:00:00  47.266945      56.622078      62.062817   \n",
       "\n",
       "   TimeGPT-lo-80  TimeGPT-lo-90  \n",
       "0      37.345340      32.855870  \n",
       "1      38.281628      32.717170  \n",
       "2      40.705810      34.119210  \n",
       "3      37.209280      33.795586  \n",
       "4      37.911810      32.471073  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timegpt_fcst_hist_ex_vars_df = nixtla_client.forecast(\n",
    "    df=df, \n",
    "    h=24, \n",
    "    level=[80, 90], \n",
    "    hist_exog_list=['Exogenous1', 'Exogenous2', 'day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6'],\n",
    ")\n",
    "timegpt_fcst_hist_ex_vars_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99be6b1b",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e7f0ecc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 231372 characters]",
      "text/plain": [
       "<Figure size 1600x700 with 4 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df[['unique_id', 'ds', 'y']], \n",
    "    timegpt_fcst_hist_ex_vars_df, \n",
    "    max_insample_length=365, \n",
    "    level=[80, 90], \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cd03e1f",
   "metadata": {},
   "source": [
    "## 3c. Forecasting electricity prices using future and historic exogenous variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96c6869c",
   "metadata": {},
   "source": [
    "A third option is to use both historic and future exogenous variables. For example, we might not have available the future information for `Exogenous1` and `Exogenous2`. In this example, we drop these variables from our future exogenous dataframe (because we assume we do not know the future value of these variables), and add them to `hist_exog_list` to be considered as historical exogenous variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81f2e577",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Using future exogenous features: ['day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6']\n",
      "INFO:nixtla.nixtla_client:Using historical exogenous features: ['Exogenous1', 'Exogenous2']\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "hist_cols = [\"Exogenous1\", \"Exogenous2\"]\n",
    "future_ex_vars_df_limited = future_ex_vars_df.drop(columns=hist_cols)\n",
    "timegpt_fcst_ex_vars_df_limited = nixtla_client.forecast(df=df, X_df=future_ex_vars_df_limited, h=24, level=[80, 90], hist_exog_list=hist_cols)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9b670da",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b392f8c0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 232292 characters]",
      "text/plain": [
       "<Figure size 1600x700 with 4 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df[['unique_id', 'ds', 'y']], \n",
    "    timegpt_fcst_ex_vars_df_limited, \n",
    "    max_insample_length=365, \n",
    "    level=[80, 90], \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a19f5eb4",
   "metadata": {},
   "source": [
    "Note that TimeGPT informs you which variables are used as historic exogenous and which are used as future exogenous."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38cd05d6",
   "metadata": {},
   "source": [
    "## 3d. Forecasting future exogenous variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4b6b0e4",
   "metadata": {},
   "source": [
    "A fourth option in case the future exogenous variables are not available is to forecast them. Below, we'll show you how we can also forecast `Exogenous1` and `Exogenous2` separately, so that you can generate the future exogenous variables in case they are not available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce9a9bce",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We read the data and create separate dataframes for the historic exogenous that we want to forecast separately.\n",
    "df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short-with-ex-vars.csv')\n",
    "df_exog1 = df[['unique_id', 'ds', 'Exogenous1']]\n",
    "df_exog2 = df[['unique_id', 'ds', 'Exogenous2']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b5abf7f",
   "metadata": {},
   "source": [
    "Next, we can use TimeGPT to forecast `Exogenous1` and `Exogenous2`. In this case, we assume these quantities can be separately forecast."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91eaa3f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Restricting input...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n",
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Restricting input...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "timegpt_fcst_ex1 = nixtla_client.forecast(df=df_exog1, h=24, target_col='Exogenous1')\n",
    "timegpt_fcst_ex2 = nixtla_client.forecast(df=df_exog2, h=24, target_col='Exogenous2')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec6cb79c",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44b4b4fd",
   "metadata": {},
   "source": [
    "We can now start creating `X_df`, which contains the future exogenous variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53cfe8e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "timegpt_fcst_ex1 = timegpt_fcst_ex1.rename(columns={'TimeGPT':'Exogenous1'})\n",
    "timegpt_fcst_ex2 = timegpt_fcst_ex2.rename(columns={'TimeGPT':'Exogenous2'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f62cec4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_df = timegpt_fcst_ex1.merge(timegpt_fcst_ex2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "697fc8e4",
   "metadata": {},
   "source": [
    "Next, we also need to add the `day_0` to `day_6` future exogenous variables. These are easy: this is just the weekday, which we can extract from the `ds` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "322e0197",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We have 7 days, for each day a separate column denoting 1/0\n",
    "for i in range(7):\n",
    "    X_df[f'day_{i}'] = 1 * (pd.to_datetime(X_df['ds']).dt.weekday == i)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8113866",
   "metadata": {},
   "source": [
    "We have now created `X_df`, let's investigate it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38ef56b7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>Exogenous1</th>\n",
       "      <th>Exogenous2</th>\n",
       "      <th>day_0</th>\n",
       "      <th>day_1</th>\n",
       "      <th>day_2</th>\n",
       "      <th>day_3</th>\n",
       "      <th>day_4</th>\n",
       "      <th>day_5</th>\n",
       "      <th>day_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>70861.410</td>\n",
       "      <td>66282.560</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>67851.830</td>\n",
       "      <td>64465.370</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>67246.660</td>\n",
       "      <td>63257.117</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>64027.203</td>\n",
       "      <td>62059.316</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>61524.086</td>\n",
       "      <td>61247.062</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 05:00:00</td>\n",
       "      <td>63054.086</td>\n",
       "      <td>62052.312</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 06:00:00</td>\n",
       "      <td>65199.473</td>\n",
       "      <td>63457.720</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 07:00:00</td>\n",
       "      <td>68285.770</td>\n",
       "      <td>65388.656</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 08:00:00</td>\n",
       "      <td>72038.484</td>\n",
       "      <td>67406.836</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 09:00:00</td>\n",
       "      <td>72821.190</td>\n",
       "      <td>68057.240</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                  ds  Exogenous1  Exogenous2  day_0  day_1  day_2  \\\n",
       "0        BE 2016-12-31 00:00:00   70861.410   66282.560      0      0      0   \n",
       "1        BE 2016-12-31 01:00:00   67851.830   64465.370      0      0      0   \n",
       "2        BE 2016-12-31 02:00:00   67246.660   63257.117      0      0      0   \n",
       "3        BE 2016-12-31 03:00:00   64027.203   62059.316      0      0      0   \n",
       "4        BE 2016-12-31 04:00:00   61524.086   61247.062      0      0      0   \n",
       "5        BE 2016-12-31 05:00:00   63054.086   62052.312      0      0      0   \n",
       "6        BE 2016-12-31 06:00:00   65199.473   63457.720      0      0      0   \n",
       "7        BE 2016-12-31 07:00:00   68285.770   65388.656      0      0      0   \n",
       "8        BE 2016-12-31 08:00:00   72038.484   67406.836      0      0      0   \n",
       "9        BE 2016-12-31 09:00:00   72821.190   68057.240      0      0      0   \n",
       "\n",
       "   day_3  day_4  day_5  day_6  \n",
       "0      0      0      1      0  \n",
       "1      0      0      1      0  \n",
       "2      0      0      1      0  \n",
       "3      0      0      1      0  \n",
       "4      0      0      1      0  \n",
       "5      0      0      1      0  \n",
       "6      0      0      1      0  \n",
       "7      0      0      1      0  \n",
       "8      0      0      1      0  \n",
       "9      0      0      1      0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac313e02",
   "metadata": {},
   "source": [
    "Let's compare it to our pre-loaded version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2a0f524",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>Exogenous1</th>\n",
       "      <th>Exogenous2</th>\n",
       "      <th>day_0</th>\n",
       "      <th>day_1</th>\n",
       "      <th>day_2</th>\n",
       "      <th>day_3</th>\n",
       "      <th>day_4</th>\n",
       "      <th>day_5</th>\n",
       "      <th>day_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>70318.0</td>\n",
       "      <td>64108.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>67898.0</td>\n",
       "      <td>62492.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>68379.0</td>\n",
       "      <td>61571.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>64972.0</td>\n",
       "      <td>60381.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>62900.0</td>\n",
       "      <td>60298.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 05:00:00</td>\n",
       "      <td>62364.0</td>\n",
       "      <td>60339.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 06:00:00</td>\n",
       "      <td>64242.0</td>\n",
       "      <td>62576.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 07:00:00</td>\n",
       "      <td>65884.0</td>\n",
       "      <td>63732.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 08:00:00</td>\n",
       "      <td>68217.0</td>\n",
       "      <td>66235.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 09:00:00</td>\n",
       "      <td>69921.0</td>\n",
       "      <td>66801.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                   ds  Exogenous1  Exogenous2  day_0  day_1  day_2  \\\n",
       "0        BE  2016-12-31 00:00:00     70318.0     64108.0    0.0    0.0    0.0   \n",
       "1        BE  2016-12-31 01:00:00     67898.0     62492.0    0.0    0.0    0.0   \n",
       "2        BE  2016-12-31 02:00:00     68379.0     61571.0    0.0    0.0    0.0   \n",
       "3        BE  2016-12-31 03:00:00     64972.0     60381.0    0.0    0.0    0.0   \n",
       "4        BE  2016-12-31 04:00:00     62900.0     60298.0    0.0    0.0    0.0   \n",
       "5        BE  2016-12-31 05:00:00     62364.0     60339.0    0.0    0.0    0.0   \n",
       "6        BE  2016-12-31 06:00:00     64242.0     62576.0    0.0    0.0    0.0   \n",
       "7        BE  2016-12-31 07:00:00     65884.0     63732.0    0.0    0.0    0.0   \n",
       "8        BE  2016-12-31 08:00:00     68217.0     66235.0    0.0    0.0    0.0   \n",
       "9        BE  2016-12-31 09:00:00     69921.0     66801.0    0.0    0.0    0.0   \n",
       "\n",
       "   day_3  day_4  day_5  day_6  \n",
       "0    0.0    0.0    1.0    0.0  \n",
       "1    0.0    0.0    1.0    0.0  \n",
       "2    0.0    0.0    1.0    0.0  \n",
       "3    0.0    0.0    1.0    0.0  \n",
       "4    0.0    0.0    1.0    0.0  \n",
       "5    0.0    0.0    1.0    0.0  \n",
       "6    0.0    0.0    1.0    0.0  \n",
       "7    0.0    0.0    1.0    0.0  \n",
       "8    0.0    0.0    1.0    0.0  \n",
       "9    0.0    0.0    1.0    0.0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "future_ex_vars_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "716d3d38",
   "metadata": {},
   "source": [
    "As you can see, the values for `Exogenous1` and `Exogenous2` are slightly different, which makes sense because we've made a forecast of these values with TimeGPT."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f78ab3ff",
   "metadata": {},
   "source": [
    "Let's create a new forecast of our electricity prices with TimeGPT using our new `X_df`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a104659d",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Using future exogenous features: ['Exogenous1', 'Exogenous2', 'day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6']\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT</th>\n",
       "      <th>TimeGPT-hi-80</th>\n",
       "      <th>TimeGPT-hi-90</th>\n",
       "      <th>TimeGPT-lo-80</th>\n",
       "      <th>TimeGPT-lo-90</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>46.987225</td>\n",
       "      <td>56.953213</td>\n",
       "      <td>61.442684</td>\n",
       "      <td>37.021236</td>\n",
       "      <td>32.531765</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>25.719133</td>\n",
       "      <td>34.580242</td>\n",
       "      <td>40.144700</td>\n",
       "      <td>16.858023</td>\n",
       "      <td>11.293568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>38.553528</td>\n",
       "      <td>45.159195</td>\n",
       "      <td>51.745792</td>\n",
       "      <td>31.947860</td>\n",
       "      <td>25.361261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>35.771927</td>\n",
       "      <td>45.787163</td>\n",
       "      <td>49.200855</td>\n",
       "      <td>25.756690</td>\n",
       "      <td>22.342999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>34.555115</td>\n",
       "      <td>43.910248</td>\n",
       "      <td>49.350986</td>\n",
       "      <td>25.199984</td>\n",
       "      <td>19.759243</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                  ds    TimeGPT  TimeGPT-hi-80  TimeGPT-hi-90  \\\n",
       "0        BE 2016-12-31 00:00:00  46.987225      56.953213      61.442684   \n",
       "1        BE 2016-12-31 01:00:00  25.719133      34.580242      40.144700   \n",
       "2        BE 2016-12-31 02:00:00  38.553528      45.159195      51.745792   \n",
       "3        BE 2016-12-31 03:00:00  35.771927      45.787163      49.200855   \n",
       "4        BE 2016-12-31 04:00:00  34.555115      43.910248      49.350986   \n",
       "\n",
       "   TimeGPT-lo-80  TimeGPT-lo-90  \n",
       "0      37.021236      32.531765  \n",
       "1      16.858023      11.293568  \n",
       "2      31.947860      25.361261  \n",
       "3      25.756690      22.342999  \n",
       "4      25.199984      19.759243  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timegpt_fcst_ex_vars_df_new = nixtla_client.forecast(df=df, X_df=X_df, h=24, level=[80, 90])\n",
    "timegpt_fcst_ex_vars_df_new.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a65798f",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2ba53a1",
   "metadata": {},
   "source": [
    "Let's create a combined dataframe with the two forecasts and plot the values to compare the forecasts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62f20711",
   "metadata": {},
   "outputs": [],
   "source": [
    "timegpt_fcst_ex_vars_df = timegpt_fcst_ex_vars_df.rename(columns={'TimeGPT':'TimeGPT-provided_exogenous'})\n",
    "timegpt_fcst_ex_vars_df_new = timegpt_fcst_ex_vars_df_new.rename(columns={'TimeGPT':'TimeGPT-forecasted_exogenous'})\n",
    "\n",
    "forecasts = timegpt_fcst_ex_vars_df[['unique_id', 'ds', 'TimeGPT-provided_exogenous']].merge(timegpt_fcst_ex_vars_df_new[['unique_id', 'ds', 'TimeGPT-forecasted_exogenous']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54fcf5cd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 246368 characters]",
      "text/plain": [
       "<Figure size 1600x700 with 4 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df[['unique_id', 'ds', 'y']], \n",
    "    forecasts, \n",
    "    max_insample_length=365, \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "019c6510",
   "metadata": {},
   "source": [
    "As you can see, we obtain a slightly different forecast if we use our forecasted exogenous variables. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
