Facebook Prophet Model with Bayesian Optimization for USD Index Prediction

.


I. INTRODUCTION
The utilization of indices to assess stock performance has been in existence since the 19th century.These indices are used to track various market segments.However, they fail to fully capture the purchasing power of the underlying currency.Therefore, if the value of the currency is not stable, the index will not provide an accurate picture of wealth changes [1].The USD Index proves valuable for traders as it allows them to track the movement of the US dollar against a group of other currencies in a single transaction.However, predicting financial market conditions has become a complex challenge.Recent research demonstrates that the application of artificial intelligence, machine learning, and neural networks possesses the capability to address non-linear and complex features of historical data, thereby generating precise accurate and useful predictions [2].
The researchers employed the Facebook Prophet method in this study due to its utilization of a straightforward and adaptable regression model.This model could produce satisfactory results using default parameters.It offers analysts the flexibility to select the appropriate components for prediction tasks and allows for convenient adjustments based on specific requirements [3].Manual hyperparameter optimization is a widely used and uncomplicated approach for this purpose.However, this approach is impractical and inefficient when dealing with complex models that require numerous adjustments and combinations [4].
The application of Bayesian Optimization can yield good results when used on datasets with non-linear, complex, and noisy characteristics.The computational requirements for identifying optimal hyperparameters can be time-consuming and can impact the performance of the model [5].In recent decades, researchers in the field of machine learning have made significant efforts.One critical task that often relies on the expertise of professionals is data pre-processing, model selection, and hyperparameter tuning.The complexity of these tasks, in some cases, has made them appear mysterious and difficult to comprehend [6].
Reference [5] applying Bayesian hyperparameter optimization to the CIFAR-10 dataset to improve model performance.With Bayesian Optimization the value of each hyperparameter can be known, thereby saving time, and improving model performance.The results showed that the error can be reduced by 6.2% when using the GPU (graphical processing unit) compared to the CPU (central processing unit) at the validation stage.Then, Reference [7] comparing the performance of prediction models with combined methods such as Long Short Term Memory and Bayesian Optimization into LSTM-BO-XGBoost, LSTM-XGBoost then LSTM and RNN (Recurrent Neural Network) methods.The results showed that the LSTM-BO-XGB combined method obtained the best performance with the evaluation results of the lowest RMSE and MAE metrics compared to the other models, then the accuracy and F1 scores were the highest than the other models.Furthermore, Reference [8] combines XGBoost and Bayesian Optimization to predict train arrival delays.In the table of performance test results in three delay scenarios, the XGB-BO combined method outperforms all other methods in the study in all delay scenarios.BO increases the accuracy of XGB predictions and improves efficiency in the parameter optimization process.
Bayesian optimization has also recently been applied in various fields to improve model performance and accuracy.In the field of classification and diagnostics, there have been developments such as the BO-SVM model for Parkinson's disease [9] and an automation system that utilizes CNN and KNN for the diagnosis of Alzheimer's [10].In the field of prediction, Bayesian optimization has been used for diverse applications including predicting jump height after the release of ice on the transmission [11], cryptocurrency price prediction [12], forecasting the number of vehicles on the road [13], predicting the collapse of transmission foundation towers [14], real-time prediction of electric load on a smart grid [15], prediction of wind energy generation [16], and a decision support prediction system for the diagnosis of gestational diabetes [17].Furthermore, Bayesian optimization has also been employed in predicting the state of health (SOH) of lithium-ion batteries [18].These articles demonstrate the success of Bayesian optimization in improving performance and accuracy across various applications.

II. METHOD
This study uses secondary data in the form of historical USD Index taken from the Yahoo Finance website which contains opening, closing, highest and lowest values daily from 2014 to 2023.

A. Prediction Model
The Facebook Data Science team developed the Facebook Prophet method based on GAN (Generalized Additive Model) [19].Designed with intuitive and wellbalanced parameters in mind without the need for indepth knowledge of the underlying model.Prophet begins by analyzing and modeling time series data using predetermined parameters, then generates predictions and evaluates them.When a problem occurs, Prophet will notify the analyst to make the necessary improvements or changes, so that the analyst can understand what has happened and how to adjust based on this input [20].Prophet requires a very short calculation time compared to other models [21].
The Prophet model consists of three key components: trend, seasonality, and holidays.These components are combined into (1).
Where () represents the trend component, which can be either piecewise linear or logistic in nature, () represents periodic changes such as daily, weekly, or yearly patterns, ℎ() represents holidays with irregular schedules, and ℇ  is the error term.

B. Optimization Method
Bayesian Optimization (BO) is an intelligent technique used to improve the performance of an objective function that takes a long time to evaluate, such as minutes or even hours.This method is particularly suitable when we want to optimize a function within a continuous range of values and with a relatively low dimensionality, typically less than 20.BO is also effective in handling random fluctuations or noise that occurs during the evaluation of the function [22] which seeks to find optimal parameter values by constructing a probability model based on previous evaluation results as in (2) [23].
Where  is all set of the hyperparameters and  represent the various set combination of the hyperparameter within ,  * is the best optimal possible hyperparameter obtained from the final optimization process, and () is the objective function needed to be optimized [24]- [25].
In the commonly used BO method, a prior Gaussian process (GP) is developed based on previous observations of () .Then, an evaluation point  is selected by maximizing the Acquisition Function (AF) derived from the posterior.At this stage, there is a set of possible values for  called .The main objective is to choose  from  in such a way that the value of () reaches the smallest or largest value.The BO method has the advantage of selecting the best value based on the evaluations performed [26], [27].By utilizing initial information about the  function and updating the acquired information, it helps reduce losses and improve the accuracy of the model.Grid Search method is inefficient to use in high-dimensional spaces, while Random Search, which is better than both, tends to only search for local optima and fails to reach the global optimum.Other evolutionary optimization methods require long training time and often yield inconsistent results.Bayesian Optimization successfully overcomes all these constraints by efficiently finding the global optimum [5].
Bayesian optimization can be thought of as like manual search.For example, when optimizing the hyperparameters of a machine learning model, one can try a set of parameters, observe the results, change one of the parameters, run it again, and compare the results.This helps determine if the search direction is correct.Bayesian optimization performs a similar process, where past performance of hyperparameters influences future decision making.In comparison, Random Search and Grid Search do not take past performance into account when determining new hyperparameters to evaluate.Therefore, Bayesian Optimization is considered as a much more efficient method [28].
1) Gaussian Process: Gaussian Process (GP) is a popular surrogate model used in Bayesian Optimization (BO) [29].The black box approach allows for good memory of all training data.In this method, Gaussian Process (GP) is initialized with evaluated values, which can be directly updated on the function being used with newer and more accurate values.By connecting the existing initial points, it is then possible to predict the function points for new test data.As a result, the model's performance significantly improves [5].
A Gaussian Process is fully determined by its mean function and covariance function, which describe the relationship between points in the GP and is noted (• )~((•),   (•,•)), with (•) as the mean function and  (•,•) as the covariance function [30].

2) Acquisition Function:
Acquisition Function (AF) is a model used in BO methods.This function helps determine which point to evaluate next in the search for the optimal value of an expensive or difficult to evaluate objective function.The acquisition function is based on a probabilistic model, such as Gaussian Process Regression, which uses information from previous evaluations of the objective function [31].There are several commonly used models of Acquisition Function in Bayesian Optimization [31]- [34] (Fig. 1). Expected Improvement (EI): Functions to estimate the likelihood of improvement compared to the best point evaluated so far. Probability of Improvement (PI): Functions to estimate the probability of improvement exceeding a certain threshold compared to the best point evaluated so far. Lower Confidence Bound (LCB): This function uses the lower confidence bound to select points that have the potential for low optimal values. Upper Confidence Bound (UCB): This function makes decisions based on the level of uncertainty in the model estimation, selecting evaluation points with the most significant level of uncertainty.
One commonly used AF is EI [30], and the formula for EI can be found in (3).

C. Model Evaluation
In this study, the model evaluation is performed using the Mean Absolute Percentage Error (MAPE) metric to assess the performance of the prediction model.
MAPE is used to compare the accuracy of predictions across different datasets.The main reason for using MAPE is that this metric is not influenced by the data scale.In other words, MAPE can provide a fair and objective comparison between forecasts in different datasets [35].The equation for evaluating the MAPE metric is shown in (4).

D. Research Steps
In the research methodology in Fig. 2, the first step involves conducting a literature review on the Facebook Prophet prediction model, Bayesian Optimization for hyperparameter tuning, and Model Evaluation using MAPE.Then, data is collected from Yahoo Finance for the USD Index from 2014 to 2023.Next, the Prophet model is applied, considering the variables ds (datestamp) and y as the target variable.This is followed by prediction using the Prophet model and the Prophet model with Bayesian Optimization.The results and performance discussion of the predictions are presented.Finally, the research concludes with a summary of findings.
Where  is the number of observations,    is the observed value at time , and    is the predicted value at time  [36].
A model should have a low MAPE value, meaning the predicted values should be close to the actual values [20].This categorization refers to Table I.

III. RESULT AND DISCUSSION
This study applies the Bayesian Optimization (BO) method to find the best hyperparameters in the prediction model.Predictions are made with various data ranges, both using BO and without BO.Subsequently, an analysis of the predicted performance of the model with and without BO is conducted.This study aims to see whether the use of BO can significantly improve the quality of predictions.

A. USD Index Data
The data retrieval was done using the Python library yfinance with the ticker 'DX-Y.NYB' from January 1, 2014, to May 18, 2023.The variable datestamp was used to capture trends and historical patterns in the data, and the closing values of the USD Index were extracted as the target variable.A sample of the data is presented in Table II.

B. Bayesian Optimization
Table III shows the hyperparameters used in the model.changepoint_prior_scale determines the priority scale in the trend, seasonality_prior_scale determines the priority scale in seasonality, interval_width sets the width of the confidence interval, changepoint_range sets the range of percentage changes in the trend, and uncertainty_samples determine the number of samples used to estimate the uncertainty in the trend and seasonality components [37]- [39].

Percentage
Forecast Evaluation Accuracy <10% The accuracy of forecasting is excellent 10-20% The accuracy of forecasting is good 20-50% The accuracy of forecasting is sufficiently good >50% The accuracy of forecasting is poor

Fig. 2 Method stages
Table IV shows the settings for optimization using BO.In this context, there are two specified parameters: n_iter and init_points, which are used to control the optimization process.The n_iter parameter is set to 25, indicating that the optimization will run for 25 iterations.During each iteration, BO will evaluate and search for the optimal combination for the model.On the other hand, the init_points parameter is set to 10, which means there are 10 initial points randomly chosen to start the optimization process.

C. Model Prediction
In the Prophet model predictions, all the testing used the same set of parameters.In this case, these parameters include general configurations commonly used in the Prophet model.In Table V, the three parameters account for the fluctuations in each season.By enabling these parameters, the model can adapt to the characteristics of each season, resulting in forecasts that are relevant to the existing seasonal patterns.
The outcomes of the optimization process utilizing BO to identify the most suitable hyperparameters are presented in Table VI.For each distinct dataset range, the optimized parameters match the ones listed in Table III.This indicates that BO systematically optimized these parameters for each dataset range, leading to optimal values that align with the underlying data patterns in each specific dataset.

D. Prediction Result
In this study, the main indicator for evaluating the prediction results of the Prophet model and the Prophet model with BO optimization is MAPE (Mean Absolute Percentage Error).Other metrics such as RMSE (Root Mean Squared Error), MSE (Mean Squared Error), MAE (Mean Absolute Error) and R2 (Coefficient of Determination) for a more comprehensive assessment of model performance.

IV. CONCLUSION
From this study, it can be concluded that the Prophet model with Bayesian Optimization is an effective approach for predicting the USD Index with more accurate metric evaluation results or higher prediction accuracy.The increased accuracy of these prediction has important implications in various contexts.The accuracy of these predictions can assist market stakeholder, investors, and financial analysts in making better decisions regarding trade and investments involving the USD.

Model Parameter Parameter Settings Across Dataset Ranges 2014-2023 2015-2023 2016-2023 2017-2023 2018-2023 2019-2023
Table VII presenting the results of the metric evaluation for the Prophet model and the Prophet model with optimization using BO are presented.The metric evaluation is conducted on predictions made using an initial dataset from 2014 to 2019 until the end of the 2023 dataset.Based on the results of the metric evaluation, it is known that the Prophet with BO model shows better performance compared to the Prophet model without BO optimization.This can be observed between the two metric evaluations result such as MAPE, RMSE, MSE, MAE and R2.Retest the predictions for the next month, evaluating the performance of the prediction model over a shorter timeframe and see if the results obtained earlier on the one-year predictions remain consistent.The results of the second test are shown in Table VIII for the model based on Bayesian Optimization hyperparameter tuning result, and Table IX presents the evaluation metrics result, which predicts the value of the USD Index for the next month.These two models have proven to be consistent by having similar performance for both the Prophet model and the Prophet model using BO.

TABLE VIII PROPHET
MODEL WITH BAYESIAN OPTIMIZATION FOR A MONTH PREDICTION