{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "!pip install -Uqq nixtla utilsforecast"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "from nixtla.utils import in_colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide \n",
    "IN_COLAB = in_colab()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    from nixtla.utils import colab_badge\n",
    "    from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fine-tuning with a specific loss function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When fine-tuning, the model trains on your dataset to tailor its predictions to your particular scenario. As such, it is possible to specify the loss function used during fine-tuning.\\\n",
    "\\\n",
    "Specifically, you can choose from:\n",
    "\n",
    "* `\"default\"` - a proprietary loss function that is robust to outliers\n",
    "* `\"mae\"` - mean absolute error\n",
    "* `\"mse\"` - mean squared error\n",
    "* `\"rmse\"` - root mean squared error\n",
    "* `\"mape\"` - mean absolute percentage error\n",
    "* `\"smape\"` - symmetric mean absolute percentage error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/tutorials/07_loss_function_finetuning.ipynb)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#| echo: false\n",
    "if not IN_COLAB:\n",
    "    load_dotenv()    \n",
    "    colab_badge('docs/tutorials/07_loss_function_finetuning')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Import packages\n",
    "First, we import the required packages and initialize the Nixtla client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from nixtla import NixtlaClient\n",
    "from utilsforecast.losses import mae, mse, rmse, mape, smape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nixtla_client = NixtlaClient(\n",
    "    # defaults to os.environ.get(\"NIXTLA_API_KEY\")\n",
    "    api_key = 'my_api_key_provided_by_nixtla'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 👍 Use an Azure AI endpoint\n",
    "> \n",
    "> To use an Azure AI endpoint, remember to set also the `base_url` argument:\n",
    "> \n",
    "> `nixtla_client = NixtlaClient(base_url=\"you azure ai endpoint\", api_key=\"your api_key\")`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| hide\n",
    "if not IN_COLAB:\n",
    "    nixtla_client = NixtlaClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fine-tune the model on a dataset using the mean absolute error (MAE).\\\n",
    "\\\n",
    "For that, we simply pass the appropriate string representing the loss function to the `finetune_loss` parameter of the `forecast` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1949-01-01</td>\n",
       "      <td>112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1949-02-01</td>\n",
       "      <td>118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1949-03-01</td>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1949-04-01</td>\n",
       "      <td>129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1949-05-01</td>\n",
       "      <td>121</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   unique_id   timestamp  value\n",
       "0          1  1949-01-01    112\n",
       "1          1  1949-02-01    118\n",
       "2          1  1949-03-01    132\n",
       "3          1  1949-04-01    129\n",
       "4          1  1949-05-01    121"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/air_passengers.csv')\n",
    "df.insert(loc=0, column='unique_id', value=1)\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Fine-tuning with Mean Absolute Error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fine-tune the model on a dataset using the Mean Absolute Error (MAE).\\\n",
    "\\\n",
    "For that, we simply pass the appropriate string representing the loss function to the `finetune_loss` parameter of the `forecast` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "timegpt_fcst_finetune_mae_df = nixtla_client.forecast(\n",
    "    df=df, \n",
    "    h=12, \n",
    "    finetune_steps=10,\n",
    "    finetune_loss='mae',   # Set your desired loss function\n",
    "    time_col='timestamp', \n",
    "    target_col='value',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 81324 characters]",
      "text/plain": [
       "<Figure size 2400x350 with 1 Axes>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nixtla_client.plot(\n",
    "    df, timegpt_fcst_finetune_mae_df, \n",
    "    time_col='timestamp', target_col='value',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, depending on your data, you will use a specific error metric to accurately evaluate your forecasting model's performance.\\\n",
    "\\\n",
    "Below is a non-exhaustive guide on which metric to use depending on your use case.\\\n",
    "\\\n",
    "**Mean absolute error (MAE)**\\\n",
    "\\\n",
    "<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{MAE}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} |y_{\\tau} - \\hat{y}_{\\tau}|\" />\n",
    "\n",
    "- Robust to outliers\n",
    "- Easy to understand\n",
    "- You care equally about all error sizes\n",
    "- Same units as your data\n",
    "\n",
    "**Mean squared error (MSE)**\\\n",
    "\\\n",
    "<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{MSE}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} (y_{\\tau} - \\hat{y}_{\\tau})^{2}\" />\n",
    "\n",
    "- You want to penalize large errors more than small ones\n",
    "- Sensitive to outliers\n",
    "- Used when large errors must be avoided\n",
    "- *Not* the same units as your data\n",
    "\n",
    "**Root mean squared error (RMSE)**\\\n",
    "\\\n",
    "<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{RMSE}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\sqrt{\\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} (y_{\\tau} - \\hat{y}_{\\tau})^{2}}\" />\n",
    "\n",
    "- Brings the MSE back to original units of data\n",
    "- Penalizes large errors more than small ones\n",
    "\n",
    "**Mean absolute percentage error (MAPE)**\\\n",
    "\\\n",
    "<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{MAPE}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} \\frac{|y_{\\tau}-\\hat{y}_{\\tau}|}{|y_{\\tau}|}\" />\n",
    "\n",
    "- Easy to understand for non-technical stakeholders\n",
    "- Expressed as a percentage\n",
    "- Heavier penalty on positive errors over negative errors\n",
    "- To be avoided if your data has values close to 0 or equal to 0\n",
    "\n",
    "**Symmmetric mean absolute percentage error (sMAPE)**\\\n",
    "\\\n",
    "<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{SMAPE}_{2}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} \\frac{|y_{\\tau}-\\hat{y}_{\\tau}|}{|y_{\\tau}|+|\\hat{y}_{\\tau}|}\" />\n",
    "\n",
    "- Fixes bias of MAPE\n",
    "- Equally senstitive to over and under forecasting\n",
    "- To be avoided if your data has values close to 0 or equal to 0\n",
    "\n",
    "With TimeGPT, you can choose your loss function during fine-tuning as to maximize the model's performance metric for your particular use case.\\\n",
    "\\\n",
    "Let's run a small experiment to see how each loss function improves their associated metric when compared to the default setting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = df[:-36]\n",
    "test = df[-36:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon. This may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n",
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon. This may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n",
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon. This may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n",
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon. This may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n",
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon. This may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n",
      "INFO:nixtla.nixtla_client:Validating inputs...\n",
      "INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
      "INFO:nixtla.nixtla_client:Inferred freq: MS\n",
      "WARNING:nixtla.nixtla_client:The specified horizon \"h\" exceeds the model horizon. This may lead to less accurate forecasts. Please consider using a smaller horizon.\n",
      "INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
     ]
    }
   ],
   "source": [
    "losses = ['default', 'mae', 'mse', 'rmse', 'mape', 'smape']\n",
    "\n",
    "test = test.copy()\n",
    "\n",
    "for loss in losses:\n",
    "    preds_df = nixtla_client.forecast(\n",
    "    df=train, \n",
    "    h=36, \n",
    "    finetune_steps=10,\n",
    "    finetune_loss=loss,\n",
    "    time_col='timestamp', \n",
    "    target_col='value')\n",
    "\n",
    "    preds = preds_df['TimeGPT'].values\n",
    "\n",
    "    test.loc[:,f'TimeGPT_{loss}'] = preds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 📘 Available models in Azure AI\n",
    ">\n",
    "> If you are using an Azure AI endpoint, please be sure to set `model=\"azureai\"`:\n",
    ">\n",
    "> `nixtla_client.forecast(..., model=\"azureai\")`\n",
    "> \n",
    "> For the public API, we support two models: `timegpt-1` and `timegpt-1-long-horizon`. \n",
    "> \n",
    "> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_id</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>value</th>\n",
       "      <th>TimeGPT_default</th>\n",
       "      <th>TimeGPT_mae</th>\n",
       "      <th>TimeGPT_mse</th>\n",
       "      <th>TimeGPT_rmse</th>\n",
       "      <th>TimeGPT_mape</th>\n",
       "      <th>TimeGPT_smape</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>108</th>\n",
       "      <td>1</td>\n",
       "      <td>1958-01-01</td>\n",
       "      <td>340</td>\n",
       "      <td>347.134094</td>\n",
       "      <td>341.933563</td>\n",
       "      <td>347.600616</td>\n",
       "      <td>347.059113</td>\n",
       "      <td>356.154938</td>\n",
       "      <td>341.958679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>1</td>\n",
       "      <td>1958-02-01</td>\n",
       "      <td>318</td>\n",
       "      <td>345.739746</td>\n",
       "      <td>343.268738</td>\n",
       "      <td>346.399963</td>\n",
       "      <td>345.678314</td>\n",
       "      <td>354.163422</td>\n",
       "      <td>343.929657</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110</th>\n",
       "      <td>1</td>\n",
       "      <td>1958-03-01</td>\n",
       "      <td>362</td>\n",
       "      <td>394.611450</td>\n",
       "      <td>390.873169</td>\n",
       "      <td>395.436646</td>\n",
       "      <td>394.636627</td>\n",
       "      <td>396.496155</td>\n",
       "      <td>392.543640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>1</td>\n",
       "      <td>1958-04-01</td>\n",
       "      <td>348</td>\n",
       "      <td>404.133545</td>\n",
       "      <td>400.997070</td>\n",
       "      <td>404.369598</td>\n",
       "      <td>403.498901</td>\n",
       "      <td>396.927185</td>\n",
       "      <td>402.459625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>112</th>\n",
       "      <td>1</td>\n",
       "      <td>1958-05-01</td>\n",
       "      <td>363</td>\n",
       "      <td>421.236542</td>\n",
       "      <td>418.793365</td>\n",
       "      <td>422.122223</td>\n",
       "      <td>421.541443</td>\n",
       "      <td>410.335663</td>\n",
       "      <td>422.161255</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     unique_id   timestamp  value  TimeGPT_default  TimeGPT_mae  TimeGPT_mse  \\\n",
       "108          1  1958-01-01    340       347.134094   341.933563   347.600616   \n",
       "109          1  1958-02-01    318       345.739746   343.268738   346.399963   \n",
       "110          1  1958-03-01    362       394.611450   390.873169   395.436646   \n",
       "111          1  1958-04-01    348       404.133545   400.997070   404.369598   \n",
       "112          1  1958-05-01    363       421.236542   418.793365   422.122223   \n",
       "\n",
       "     TimeGPT_rmse  TimeGPT_mape  TimeGPT_smape  \n",
       "108    347.059113    356.154938     341.958679  \n",
       "109    345.678314    354.163422     343.929657  \n",
       "110    394.636627    396.496155     392.543640  \n",
       "111    403.498901    396.927185     402.459625  \n",
       "112    421.541443    410.335663     422.161255  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#| hide\n",
    "test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! We have predictions from TimeGPT using all the different loss functions. We can evaluate the performance using their associated metric and measure the improvement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loss_fct_dict = {\n",
    "    \"mae\": mae,\n",
    "    \"mse\": mse,\n",
    "    \"rmse\": rmse,\n",
    "    \"mape\": mape,\n",
    "    \"smape\": smape\n",
    "}\n",
    "\n",
    "pct_improv = []\n",
    "\n",
    "for loss in losses[1:]:\n",
    "    evaluation = loss_fct_dict[f'{loss}'](test, models=['TimeGPT_default', f'TimeGPT_{loss}'], id_col='unique_id', target_col='value')\n",
    "    pct_diff = (evaluation['TimeGPT_default'] - evaluation[f'TimeGPT_{loss}']) / evaluation['TimeGPT_default'] * 100\n",
    "    pct_improv.append(round(pct_diff, 2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mae</th>\n",
       "      <th>mse</th>\n",
       "      <th>rmse</th>\n",
       "      <th>mape</th>\n",
       "      <th>smape</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Metric improvement (%)</th>\n",
       "      <td>8.54</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.64</td>\n",
       "      <td>31.02</td>\n",
       "      <td>7.36</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         mae   mse  rmse   mape  smape\n",
       "Metric improvement (%)  8.54  0.31  0.64  31.02   7.36"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = {\n",
    "    'mae': pct_improv[0].values,\n",
    "    'mse': pct_improv[1].values,\n",
    "    'rmse': pct_improv[2].values,\n",
    "    'mape': pct_improv[3].values,\n",
    "    'smape': pct_improv[4].values\n",
    "}\n",
    "\n",
    "metrics_df = pd.DataFrame(data)\n",
    "metrics_df.index = ['Metric improvement (%)']\n",
    "\n",
    "metrics_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the table above, we can see that using a specific loss function during fine-tuning will improve its associated error metric when compared to the default loss function.\\\n",
    "\\\n",
    "In this example, using the MAE as the loss function improves the metric by 8.54% when compared to using the default loss function.\\\n",
    "\\\n",
    "That way, depending on your use case and performance metric, you can use the appropriate loss function to maximize the accuracy of the forecasts."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
