4_data_requirements.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Requirements\n",
    "\n",
    "> This section explains the data requirements for `TimeGPT`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "from nixtla.utils import colab_badge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/getting-started/5_data_requirements.ipynb)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#| echo: false\n",
    "colab_badge('docs/getting-started/5_data_requirements')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`TimeGPT` accepts `pandas` and `polars` dataframes in [long format](https://www.theanalysisfactor.com/wide-and-long-data/#comments) with the following necessary columns: \n",
    "\n",
    "- `ds` (timestamp): timestamp in format `YYYY-MM-DD` or `YYYY-MM-DD HH:MM:SS`. \n",
    "- `y` (numeric): The target variable to forecast. \n",
    "\n",
    "(Optionally, you can also pass a DataFrame without the `ds` column as long as it has DatetimeIndex)\n",
    "\n",
    "`TimeGPT` also works with distributed dataframes like `dask`, `spark` and `ray`. \n",
    "\n",
    "You can also include exogenous features in the DataFrame as additional columns. For more information, follow this [tutorial](https://docs.nixtla.io/docs/tutorials-exogenous_variables).\n",
    "\n",
    "Below is an example of a valid input dataframe for `TimeGPT`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1949-01-01</td>\n",
       "      <td>112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1949-02-01</td>\n",
       "      <td>118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1949-03-01</td>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1949-04-01</td>\n",
       "      <td>129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1949-05-01</td>\n",
       "      <td>121</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    timestamp  value\n",
       "0  1949-01-01    112\n",
       "1  1949-02-01    118\n",
       "2  1949-03-01    132\n",
       "3  1949-04-01    129\n",
       "4  1949-05-01    121"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd \n",
    "\n",
    "df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/air_passengers.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that in this example, the `ds` column is named `timestamp` and the `y` column is named `value`. You can either:\n",
    "\n",
    "1. Rename the columns to `ds` and `y`, respectively, or\n",
    "\n",
    "2. Keep the current column names and specify them when using any method from the `NixtlaClient` class with the `time_col` and `target_col` arguments. \n",
    "\n",
    "For example, when using the `forecast` method from the `NixtlaClient` class, you must instantiate the class and then specify the columns names as follows. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nixtla import NixtlaClient\n",
    "\n",
    "nixtla_client = NixtlaClient(\n",
    "    api_key = 'my_api_key_provided_by_nixtla'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "nixtla_client = NixtlaClient()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Querying model metadata...\n",
      "INFO:nixtla.nixtla_client:Restricting input...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1961-01-01</td>\n",
       "      <td>437.83792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1961-02-01</td>\n",
       "      <td>426.06270</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1961-03-01</td>\n",
       "      <td>463.11655</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1961-04-01</td>\n",
       "      <td>478.24450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1961-05-01</td>\n",
       "      <td>505.64648</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   timestamp    TimeGPT\n",
       "0 1961-01-01  437.83792\n",
       "1 1961-02-01  426.06270\n",
       "2 1961-03-01  463.11655\n",
       "3 1961-04-01  478.24450\n",
       "4 1961-05-01  505.64648"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fcst = nixtla_client.forecast(df=df, h=12, time_col='timestamp', target_col='value')\n",
    "fcst.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, the `NixtlaClient` is infereing the frequency, but you can explicitly specify it with the `freq` argument.\n",
    "\n",
    "\n",
    "To learn more about how to instantiate the `NixtlaClient` class, refer to the [TimeGPT Quickstart](https://docs.nixtla.io/docs/getting-started-timegpt_quickstart)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiple Series "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you're working with multiple time series, make sure that each series has a unique identifier. You can name this column `unique_id` or specify its name using the `id_col` argument when calling any method from the `NixtlaClient` class. This column should be a string, integer, or category.\n",
    "\n",
    "In this example, we have five series representing hourly electricity prices in five different markets. The columns already have the default names, so it's unnecessary to specify the `id_col`, `time_col`, or `target_col` arguments. If your columns have different names, specify these arguments as required."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 00:00:00</td>\n",
       "      <td>70.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 01:00:00</td>\n",
       "      <td>37.10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 02:00:00</td>\n",
       "      <td>37.10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 03:00:00</td>\n",
       "      <td>44.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 04:00:00</td>\n",
       "      <td>37.10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                   ds      y\n",
       "0        BE  2016-10-22 00:00:00  70.00\n",
       "1        BE  2016-10-22 01:00:00  37.10\n",
       "2        BE  2016-10-22 02:00:00  37.10\n",
       "3        BE  2016-10-22 03:00:00  44.75\n",
       "4        BE  2016-10-22 04:00:00  37.10"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Querying model metadata...\n",
      "INFO:nixtla.nixtla_client:Restricting input...\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>45.190582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>43.244987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>41.958897</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>39.796680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>39.204865</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                  ds    TimeGPT\n",
       "0        BE 2016-12-31 00:00:00  45.190582\n",
       "1        BE 2016-12-31 01:00:00  43.244987\n",
       "2        BE 2016-12-31 02:00:00  41.958897\n",
       "3        BE 2016-12-31 03:00:00  39.796680\n",
       "4        BE 2016-12-31 04:00:00  39.204865"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fcst = nixtla_client.forecast(df=df, h=24) # use id_col, time_col and target_col here if needed. \n",
    "fcst.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When working with a large number of time series, consider using a [distributed computing framework](https://docs.nixtla.io/docs/tutorials-computing_at_scale) to handle the data efficiently. `TimeGPT` supports frameworks such as [Spark](https://docs.nixtla.io/docs/tutorials-spark), [Dask](https://docs.nixtla.io/docs/tutorials-dask), and [Ray](https://docs.nixtla.io/docs/tutorials-ray)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exogenous Variables "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`TimeGPT` also accepts exogenous variables. You can add exogenous variables to your dataframe by including additional columns after the `y` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>Exogenous1</th>\n",
       "      <th>Exogenous2</th>\n",
       "      <th>day_0</th>\n",
       "      <th>day_1</th>\n",
       "      <th>day_2</th>\n",
       "      <th>day_3</th>\n",
       "      <th>day_4</th>\n",
       "      <th>day_5</th>\n",
       "      <th>day_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 00:00:00</td>\n",
       "      <td>70.00</td>\n",
       "      <td>57253.0</td>\n",
       "      <td>49593.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 01:00:00</td>\n",
       "      <td>37.10</td>\n",
       "      <td>51887.0</td>\n",
       "      <td>46073.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 02:00:00</td>\n",
       "      <td>37.10</td>\n",
       "      <td>51896.0</td>\n",
       "      <td>44927.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 03:00:00</td>\n",
       "      <td>44.75</td>\n",
       "      <td>48428.0</td>\n",
       "      <td>44483.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-10-22 04:00:00</td>\n",
       "      <td>37.10</td>\n",
       "      <td>46721.0</td>\n",
       "      <td>44338.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                   ds      y  Exogenous1  Exogenous2  day_0  day_1  \\\n",
       "0        BE  2016-10-22 00:00:00  70.00     57253.0     49593.0    0.0    0.0   \n",
       "1        BE  2016-10-22 01:00:00  37.10     51887.0     46073.0    0.0    0.0   \n",
       "2        BE  2016-10-22 02:00:00  37.10     51896.0     44927.0    0.0    0.0   \n",
       "3        BE  2016-10-22 03:00:00  44.75     48428.0     44483.0    0.0    0.0   \n",
       "4        BE  2016-10-22 04:00:00  37.10     46721.0     44338.0    0.0    0.0   \n",
       "\n",
       "   day_2  day_3  day_4  day_5  day_6  \n",
       "0    0.0    0.0    0.0    1.0    0.0  \n",
       "1    0.0    0.0    0.0    1.0    0.0  \n",
       "2    0.0    0.0    0.0    1.0    0.0  \n",
       "3    0.0    0.0    0.0    1.0    0.0  \n",
       "4    0.0    0.0    0.0    1.0    0.0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short-with-ex-vars.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using exogenous variables, you also need to provide its future values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>Exogenous1</th>\n",
       "      <th>Exogenous2</th>\n",
       "      <th>day_0</th>\n",
       "      <th>day_1</th>\n",
       "      <th>day_2</th>\n",
       "      <th>day_3</th>\n",
       "      <th>day_4</th>\n",
       "      <th>day_5</th>\n",
       "      <th>day_6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>70318.0</td>\n",
       "      <td>64108.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>67898.0</td>\n",
       "      <td>62492.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>68379.0</td>\n",
       "      <td>61571.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>64972.0</td>\n",
       "      <td>60381.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>62900.0</td>\n",
       "      <td>60298.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                   ds  Exogenous1  Exogenous2  day_0  day_1  day_2  \\\n",
       "0        BE  2016-12-31 00:00:00     70318.0     64108.0    0.0    0.0    0.0   \n",
       "1        BE  2016-12-31 01:00:00     67898.0     62492.0    0.0    0.0    0.0   \n",
       "2        BE  2016-12-31 02:00:00     68379.0     61571.0    0.0    0.0    0.0   \n",
       "3        BE  2016-12-31 03:00:00     64972.0     60381.0    0.0    0.0    0.0   \n",
       "4        BE  2016-12-31 04:00:00     62900.0     60298.0    0.0    0.0    0.0   \n",
       "\n",
       "   day_3  day_4  day_5  day_6  \n",
       "0    0.0    0.0    1.0    0.0  \n",
       "1    0.0    0.0    1.0    0.0  \n",
       "2    0.0    0.0    1.0    0.0  \n",
       "3    0.0    0.0    1.0    0.0  \n",
       "4    0.0    0.0    1.0    0.0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "future_ex_vars_df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short-future-ex-vars.csv')\n",
    "future_ex_vars_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: h\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Using future exogenous features: ['Exogenous1', 'Exogenous2', 'day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6']\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>ds</th>\n",
       "      <th>TimeGPT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 00:00:00</td>\n",
       "      <td>51.632830</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 01:00:00</td>\n",
       "      <td>45.750877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 02:00:00</td>\n",
       "      <td>39.650543</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 03:00:00</td>\n",
       "      <td>34.000072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BE</td>\n",
       "      <td>2016-12-31 04:00:00</td>\n",
       "      <td>33.785370</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_id                  ds    TimeGPT\n",
       "0        BE 2016-12-31 00:00:00  51.632830\n",
       "1        BE 2016-12-31 01:00:00  45.750877\n",
       "2        BE 2016-12-31 02:00:00  39.650543\n",
       "3        BE 2016-12-31 03:00:00  34.000072\n",
       "4        BE 2016-12-31 04:00:00  33.785370"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fcst = nixtla_client.forecast(df=df, X_df=future_ex_vars_df, h=24)\n",
    "fcst.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To learn more about how to use exogenous variables with `TimeGPT`, consult the [Exogenous Variables](https://docs.nixtla.io/docs/tutorials-exogenous_variables) tutorial. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Important Considerations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using `TimeGPT`, the data cannot contain missing values. This means that for every series, there should be no gaps in the timestamps and no missing values in the target variable. \n",
    "\n",
    "For more, please refer to the tutorial on [Dealing with Missing Values in TimeGPT](https://docs.nixtla.io/docs/tutorials-dealing_with_missing_values_in_timegpt). \n",
    "\n",
    "### Minimum Data Requirements (for AzureAI)\n",
    "\n",
    "`TimeGPT` currently supports any amount of data for generating point forecasts. That is, the minimum size per series to expect results from this call `nixtla_client.forecast(df=df, h=h, freq=freq)` is one, regardless of the frequency.\n",
    "\n",
    "For Azure AI, when  using the arguments `level`, `finetune_steps`, `X_df` (exogenous variables), or `add_history`, the API requires a minimum number of data points depending on the frequency. Here are the minimum sizes for each frequency:\n",
    "\n",
    "<div align=\"center\">\n",
    "\n",
    "| Frequency                | Minimum Size |\n",
    "|--------------------------|--------------|\n",
    "| Hourly and subhourly (e.g., \"H\", \"min\", \"15T\") | 1008         |\n",
    "| Daily (\"D\")              | 300          |\n",
    "| Weekly (e.g., \"W-MON\",..., \"W-SUN\") | 64           |\n",
    "| Monthly and other frequencies (e.g., \"M\", \"MS\", \"Y\") | 48      |\n",
    "\n",
    "</div>\n",
    "\n",
    "For cross-validation, you need to consider these numbers as well as the forecast horizon (`h`), the number of windows (`n_windows`), and the gap between windows (`step_size`). Thus, the minimum number of observations per series in this case would be determined by the following relationship:\n",
    "\n",
    "<div align=\"center\">\n",
    "\n",
    "Minimum number described previously + h + step_size + (n_windows - 1)\n",
    "\n",
    "</div>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}