Artificial intelligence-based stock market price prediction, a review

1. Introduction

The stock market, a critical component of the global financial system, reflects economic health, and serves as a platform for investment and wealth creation. The accurate prediction of stock market price trends is essential for investors, financial institutions, and policymakers to optimize portfolio management, mitigate risks, and make informed decisions. However, the nonlinear, dynamic, and stochastic nature of financial markets poses significant challenges to traditional forecasting methods, such as Autoregressive Integrated Moving Average Artificial Neural networks (ARIMA) and GARCH [1, 2]. These models often struggle to capture the complex dependencies and rapid fluctuations inherent to financial data.

Recent advancements in Artificial Intelligence (AI) have revolutionized stock market predictions by introducing advanced computational models capable of analyzing vast datasets and uncovering intricate patterns. Machine Learning (ML) and Deep Learning (DL) techniques have demonstrated superior performance in handling nonlinearity and large-scale data compared to traditional statistical approaches [3, 4]. Models such as Long Short-Term Memory (LSTM) networks capture temporal dependencies in sequential data [5], whereas hybrid frameworks such as K-means LSTM enhance the predictive accuracy by integrating clustering with temporal modeling [6].

In addition, Generative Adversarial Networks (GANs) address challenges such as data sparsity by generating synthetic datasets to improve model robustness against volatility [7]. Sentiment analysis models such as Stock Senti WordNet (SSWN) leverage social media data to incorporate public sentiment into predictive frameworks, further enhancing short-term forecasting accuracy [8]. Explainable AI frameworks such as Hybrid Quantum Neural Networks (HQNN) integrate sentiment analysis with quantum computing principles to improve interpretability while maintaining high accuracy [9].

This review aims to provide a comprehensive analysis of AI-based methodologies for stock market price trend prediction by evaluating existing models, datasets, and performance metrics and identifying research gaps and future opportunities.

The remainder of this paper is organized as follows. Section 2 highlights the significance of the topic and Section 3 describes the problem statement. Section 4 outlines the objectives and Section 5 provides a comprehensive literature review. Section 6 details the taxonomy employed and Section 7 discusses the datasets used. Section 8 presents a comparative analysis of various algorithms, and Section 9 identifies gaps and challenges. Section 10 explores future research directions and Section 11 concludes the paper.

2. Significance of the topic

The study of stock market price trend prediction holds immense importance because of its profound impact on investment strategies, risk management, and economic decision making. The integration of AI with financial analytics has transformed the traditional forecasting methods. The significance of this study is analyzed through its economic, technological, and practical implications.

1) Economic Impact: Stock market predictions influence investment decisions at the individual and institutional levels, shaping the global financial ecosystem.

– Investment Decisions: Accurate predictions empower investors to make informed decisions and enhance wealth creation. Institutional investors use predictive models to optimize portfolio allocation.

– Economic Stability: Stock trends indicate economic performance. A bullish market signals investor confidence and stimulates economic growth through corporate investment [10].

– Global Financial Impact: The adoption of AI in financial forecasting is growing rapidly. The financial services sector invested $35 billion in AI projects in 2023, with the global AI in finance market projected to grow at a CAGR of 30.6 %, reaching $190 billion [10] by 2030. Even marginal improvements in the prediction accuracy can result in significant financial gains.

2) Technological Advancements: AI integration has transformed stock market prediction beyond traditional forecasting models.

– Revolutionizing Traditional Models: Traditional models, such as ARIMA and GARCH, cannot handle nonlinear relationships and large datasets effectively. AI models, including LSTM networks, transformer architectures, and hybrid approaches, capture complex financial patterns and temporal dependencies in the financial data.

– Enhanced Predictive Accuracy: AI models show superior accuracy compared with traditional methods. Techniques such as hierarchical frequency decomposition combined with deep learning have shown significant improvements in capturing complex financial patterns and improving prediction accuracy [11]. LSTMs achieve 72 % accuracy in price movement predictions, whereas CNN-based models achieve 90 % accuracy [10] for trend identification.

– Real-Time Analysis: AI-based forecasting enables faster and more accurate real-time decision making, which is critical in high-frequency trading. Transformer-based models reduce prediction latency by 57 % [10] and enhance trading efficiency.

3) Risk Management: Accurate stock market predictions help mitigate risks in volatile financial markets.

– Volatility Mitigation: Financial markets are inherently volatile; as demonstrated during the COVID-19 pandemic, the CBOE Volatility Index spiked by 43.2 % in 2020, highlighting significant market uncertainty. AI-driven models can help to anticipate market movements and implement hedging strategies to reduce losses.

– Portfolio optimization: AI models help fund managers dynamically adjust portfolios based on predicted market conditions. Hybrid models such as SeroFAM with Genetic Algorithms improve portfolio returns by optimizing trading strategies. This approach reduces exposure to high-risk assets during downturns while capitalizing on growth opportunities.

– Crisis Prevention: Early detection of potential market crashes through predictive analytics helps policymakers take preemptive measures to stabilize markets and protect investments.

4) Data utilization: AI-driven stock market prediction leverages historical and real-time data to enhance the accuracy.

– Large-Scale Historical Data: AI models use historical datasets to identify the trends and patterns often overlooked by traditional models.

– Technical Indicators: The inclusion of technical indicators such as moving averages, RSI, ADX, Bollinger Bands, CMO, and MACD in predictive models enhances their ability to analyze market behavior and improve forecasting accuracy.

– Social Media Sentiment Analysis: AI models, such as Stock Senti WordNet, analyze social media sentiment and incorporate sentiment data to improve the prediction accuracy by 18% for volatile stocks.

Multisource Data Integration: Hybrid GAN-LSTM or Transformer architectures combine diverse data sources, such as historical prices, macroeconomic indicators, and real-time news sentiment for accurate forecasts.

The significance of stock market price trend prediction lies in its ability to influence investment decisions, mitigate risks, and leverage technological advancements to revolutionize financial forecasting. AI models provide unparalleled predictive accuracy and real-time decision-making capabilities by utilizing large-scale historical data, technical indicators, and social media sentiments. As global adoption of AI in finance grows, this field continues to drive innovation, reshape financial markets, and contribute to economic prosperity. However, challenges such as data noise, ethical considerations, and model scalability must be addressed to ensure equitable and transparent application of AI in financial forecasting.

3. Problem statement

The problems in recent research analytics of stock market forecasting are defined and determined in this section.

1) Algorithm and hyperparameter selection: Choosing the right algorithm and tuning its hyperparameters are complex and often require domain expertise and extensive experimentation.

2) Black Box Nature of Models: Many deep learning models operate as “black boxes, ” making it difficult for stakeholders to interpret how predictions are made. This lack of transparency reduces the trust in AI-driven systems.

3) Overfitting and generalization: Traditional and some ML models tend to overfit historical data and perform well on training sets but poorly on unseen or future datasets.

4) Feature Selection Complexity: Identifying the most predictive features (e.g., historical prices, volume, sentiment, and economic indicators) is challenging because of the dynamic and nonlinear nature of financial markets. Irrelevant or redundant features can degrade the model performance.

5) Regime Shifts and Adaptability: Financial markets are highly dynamic and undergo frequent regime shifts owing to economic crises, policy changes, or global events. Many existing models have struggled to adapt to these changes.

6) Data Quality Issues: Stock market datasets are often incomplete, noisy, or inconsistent. However, ML/DL models require high-quality and clean data for reliable prediction.

7) Nonstationarity of Financial Data: Stock prices exhibit changes in the mean and variance over time, making it difficult to model them using static assumptions.

8) Market volatility and nonlinearity: Stock prices are influenced by a wide range of factors, including macroeconomic, microeconomic, and geopolitical factors, which are often nonlinear, interdependent, and multifaceted patterns and trends, making modeling and prediction difficult.

9) Ethical and Regulatory Concerns: The use of AI in finance raises ethical concerns regarding fairness, accountability, and potential misuse (e.g., market manipulation). Regulatory frameworks that address these concerns have evolved.

4. Literature review

This section provides a comprehensive review of the recent advancements in artificial intelligence (AI)-based methodologies for predicting stock market price trends. This discussion highlights the key approaches to address forecasting challenges, including machine learning (ML) and deep learning (DL) models and their applications. By examining state-of-the-art techniques, this review highlights their strengths, limitations, and contributions to improving prediction accuracy and decision making in dynamic financial markets.

Polamuri et al. [3] implemented a “Multi-Model Generative Adversarial Network Hybrid Prediction Algorithm (MMGAN-HPA)” to effectively forecast stock prices. This method fine-tunes the hyperparameters using Bayesian optimization with reinforcement learning. The proposed method combines Generative Adversarial Networks (GANs) with traditional machine learning models to address market volatility and data noise. The GAN-related approach, with its generator and discriminator, effectively handles time-series stock data to improve the predictions. GANs generate synthetic data for training, whereas hybridization with ARIMA and LSTM improves predictive accuracy. MMGAN-HPA outperformed the standalone models and traditional GANs by reducing the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). This method excels at handling nonlinear market patterns and provides robust predictions. Although this study demonstrates the scalability of the model across different stock markets, its limitations include reliance on hyperparameter tuning, and increased computational requirements. The authors conclude that MMGAN-HPA has significant potential for accurate stock market forecasting.

Bandhu et al. [12] introduced a real-time stock price prediction framework that integrates stream processing with deep learning to handle high-velocity financial data. The model uses Apache Kafka for stream processing and combines LSTM and GRU to capture the temporal dependencies of the stock prices. This study focuses on improving prediction accuracy while ensuring low-latency processing for real-time applications. A comparative analysis of the traditional models demonstrated superior performance in terms of the MAE and RMSE. The framework was tested on real-time stock data streams, thereby demonstrating its scalability to market conditions. This study emphasizes the importance of preprocessing techniques to enhance the model performance. The ability of the framework to fetch complex dependencies leads to an increased stock price forecasting accuracy. Despite its advantages, the model is limited by the vanishing gradient problem, which affects prediction accuracy. However, the computational complexity and infrastructure requirements are drawbacks. The study concludes that this approach is effective for real-time financial decision-making.

Gandhudi et al. [9] implemented an Explainable Hybrid quantum neural network (HQNN) framework to study the effects of social media sentiments on stock market price predictions. This method uses a hybrid genetic-based quantum neural network for the stock price analysis and data fusion of transformers. The combination of LSTM neural networks and quantum computing principles helps identify complex patterns in the data for better predictions. FinBERT is used to classify tweets as positive, negative, or neutral sentiments in the HQNN model. The results show that tweet sentiment improves the stock price prediction accuracy by 18 %, especially for volatile stocks. SHAP values were used to interpret the model predictions and identify the key influencing factors. The HQNN outperforms classical neural networks in terms of speed and accuracy for large datasets. However, challenges remain in terms of quantum computing hardware scalability and real-world application. This method uses the average sentiment scores of tweets for the data combinations. The study concludes that HQNNs are promising for integrating social media data into financial forecasting, while maintaining the transparency of the model.

Gülmez [13] introduced an improved deep Long Short-Term Memory (LSTM) network using the Artificial Rabbits Optimization Algorithm (ARO) model (LSTM-ARO). The ARO algorithm optimizes key LSTM parameters, such as the learning rate, layers, and neurons, to improve prediction accuracy and reduce overfitting. The hidden layers evaluate the data, and the output is calculated using the current inputs. The hidden state and cell input help forecast stock prices, with ARO using two steps for less error forecasting than traditional models. The built-in LSTM parameters help the ARO find the best architecture for prediction. The model demonstrated superior performance on multiple stock datasets compared to traditional LSTM and other optimization models, such as particle swarm optimization (PSO) and Genetic Algorithms (GA), achieving a lower Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE). The study shows ARO’s robustness of ARO in navigating complex optimization landscapes and handling nonlinear patterns in stock data. However, the deep LSTM model with optimization is computationally expensive, and ARO exhibits slow convergence and an inefficient search. This hybrid approach is promising for advanced financial forecasting applications.

Chandola et al. [5] introduced a framework for predicting stock price directional movements using deep-learning models. The hybrid model combines Word2Vec for feature extraction and LSTM for temporal dependency. This study uses historical stock price data and technical indicators such as the RSI, MACD, and moving averages. The model achieves high accuracy in directional forecasting, outperforming traditional machine learning models such as Support Vector Machine (SVM) and Random Forests. The authors addressed class imbalance by using SMOTE. Limitations include computational intensity and potential overfitting, with multisource and multifrequency factors complicating neural-network management. This framework is effective for directional stock-price predictions.

Sangeetha et al. [14] presented an Evaluated Linear Regression Based Machine Learning (ELR-ML) algorithm for forecasting stock market trends. The method uses Ordinary Least Squares optimization to minimize prediction variability. This study incorporates historical price data and technical indicators such as moving averages and Bollinger Bands. ELR achieved competitive results compared to Decision Trees and Support Vector Machines, particularly in short-term predictions. The simplicity and interpretability of the model render it suitable for applications that require limited computational resources. The performance metrics showed consistent results across the datasets, although their linear nature limited their ability to capture the complex relationships. The study concludes that the ELR is effective for quick financial forecasting, especially for small-scale investors.

Albahli et al. [8] implemented Stock Senti WordNet (SSWN), which adds sentiment analysis to stocks to help traders make decisions. This framework uses real-time Twitter data and Natural Language Processing (NLP) to analyze sentiment from tweets, categorizing them as positive, negative, or neutral, generating sentiment scores, and integrating them with historical stock data to enhance prediction accuracy. These sentiment scores quantify market feelings about specific stocks, and can be used for news stories and earnings reports. The SSWN utilizes the Random Forest and SVM algorithms, and the results show that Twitter sentiment improves the model’s predictive capability. Sentiment research can help minimize risk by avoiding stocks through dropping sentiments and diversifying assets based on market sentiment. The authors emphasized preprocessing steps to improve the data quality. These limitations include data scope, sentiment accuracy, context sensitivity, and integration challenges, particularly because of the dynamic nature of social media. Despite these limitations, the model shows a strong potential for short-term stock prediction, concluding that real-time social media sentiment analysis with machine learning provides valuable financial market insights.

Zhang and Chen [4] introduced a two-stage stock price prediction model combining Variational Mode Decomposition (VMD) with ensemble machine learning techniques. In the first stage, VMD decomposes historical stock price data into multiple intrinsic mode functions (IMFs), isolating frequency components and reducing noise. Support Vector Machine Regression (SVR) was used to find a function that minimises prediction errors within a tolerance margin while predicting the end values of the input features. SVR uses kernel functions to handle the nonlinear relationships between features and objective values. In the second stage, the IMFs serve as inputs to an ensemble learning model comprising Gradient Boosting Decision Trees (GBDT), random forests (RF), and Extreme Gradient Boosting (XGBoost) for predictions. An Extreme Learning Machine (ELM) generates excellent generalizations by improving the base-learning speed. The authors showed that this approach improves the prediction accuracy and stability compared to single-model methods, evaluated using the Mean Absolute Percentage Error (MAPE) and Root Mean Squared Error (RMSE). This study highlights VMD's importance of VMD in addressing nonstationary and noisy stock market data. The ensemble model captured both the linear and nonlinear patterns. However, SVR faces challenges in hyperparameter tuning, computational complexity, and kernel selection, whereas ELM has limited flexibility and the risk of overfitting. The complexity of the model may limit its scalability. The results confirm that the proposed method outperforms baseline models, offering a robust framework for stock price forecasting.

Diqi et al. [7] implemented the StockGAN framework, a Generative Adversarial Networks (GAN) method, to provide the best accuracy in stock market forecasting. The model uses GAN's ability to learn complex data distributions through a generator to simulate stock price patterns and a discriminator to distinguish between actual and generated data. The generators and discriminators were trained using argumentative learning methods to improve the model and to defend against overfitting. Historical stock prices from major markets were used as inputs for preprocessing to improve data quality. The authors compared StockGAN with traditional models such as ARIMA and LSTM and demonstrated its superior accuracy in capturing nonlinear relationships in financial data. Evaluation metrics, including Mean Squared Error (MSE) and R-squared (R²), confirmed its effectiveness. GAN offer advantages such as generating realistic data, enhancing training with synthetic data, and modeling complex distributions. Although GANs require careful hyperparameter tuning to avoid mode collapse, this study demonstrates the model’s robustness against market volatility. The model has limitations including high computational demands and training instability. The study concludes that StockGAN’s dynamic training process and its effectiveness in unsupervised learning make it a powerful tool for machine learning and financial forecasting in volatile markets.

Leon Lai Xiang Yeo a et al. [15] introduced dynamic portfolio rebalancing with lag-optimized trading indicators using SeroFAM and genetic algorithms. This study explores portfolio rebalancing by integrating Self-Regulating Fuzzy Associative Memory (SeroFAM) with genetic algorithms (GAs) to optimize trading indicators. The SeroFAM model adaptively learns and adjusts to market dynamics by addressing the lag in the traditional trading indicators. Genetic algorithms were used to fine-tune the indicator parameters for optimal performance under diverse market conditions. The methodology tested on financial datasets demonstrates significant improvements in portfolio returns compared to static rebalancing strategies. Key metrics such as the Sharpe Ratio and Cumulative Returns highlight the effectiveness of this hybrid approach in managing risk while maximizing profitability. The authors emphasize the interpretability of SeroFAM for market trends and decision making. However, they noted the computational challenges of GA optimization and scalability for larger portfolios. This study contributes to the development of adaptive AI-based portfolio-management strategies.

T. Swathi et al. introduced Teaching and Learning Based Optimization model with Long Short-Term Memory (TLBO-LSTM), a hybrid model combining Teaching-Learning-Based Optimization (TLBO) and Long Short-Term Memory (LSTM) networks for stock price prediction using Twitter sentiment analysis. The model integrates sentiment scores from tweets with historical stock price data to improve the prediction accuracy. TLBO optimizes LSTM key hyperparameters, such as the learning rate and the number of layers and hidden units, by leveraging TLBO’s global optimization capabilities, eliminating manual tuning, and improving generalization for nonlinear and nonstationary financial datasets. The model processes historical stock prices, technical indicators, and sentiment scores through the teaching phase (global search for optimal hyperparameters) and the learning phase (local refinement). The optimized LSTM captured the temporal dependencies in the sequential data. The LSTM network was optimized using a metaheuristic algorithm Adam optimizer to fine-tune the hyperparameters, reduce the training time, and improve computational efficiency. The optimizer is well suited for large datasets, reducing the training time and enhancing the convergence rate, while improving the computational efficiency. Past gradients have been incorporated to achieve these improvements. The experimental results showed superior performance over ARIMA and standalone LSTMs with higher accuracy and lower error rates. However, TLBO-LSTM has limitations including computational complexity, parameter sensitivity, overfitting risk, and challenges with noisy tweets. This study demonstrates the importance of social media sentiment in market movements, and the potential of combining deep learning with alternative data sources for financial forecasting. A Generative Adversarial Network with Long Short-Term Memory (GAN-LSTM) combines GANs with LSTMs to address data imbalance by generating synthetic datasets for training to enhance the robustness against noisy data, thereby offering complementary solutions for stock market prediction, whereas TLBO-LSTM focuses on hyperparameter optimization.

Chen et al. [6] implemented a K-means-LSTM hybrid model to forecast the stock prices of Chinese commercial banks. The approach begins with K-means clustering, which groups stocks with similar price patterns, thereby reducing data complexity and improving the model focus. These clustered datasets were fed into a Long Short-Term Memory (LSTM) network, which captured the temporal dependencies and nonlinear trends in stock prices. This model can be used for static and dynamic stock price forecasting of a wide range of equities with different tendencies. By clustering stocks with similar price movements, tailored forecasting using cluster-specific models improves prediction accuracy by training the LSTM on more homogeneous data. By concentrating on pertinent patterns inside each cluster, this method enables LSTM models to accurately represent intricate temporal relationships in the stock values. The hybrid model was benchmarked against standalone LSTM and traditional machine learning methods, showing improved prediction accuracy measured using the Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). The authors highlighted the model's scalability for large datasets but noted challenges in selecting optimal K-means parameters, determining the number of clusters, and potential LSTM overfitting. The study concludes that the K-means-LSTM hybrid method is promising for stock price forecasting, particularly in sector-specific scenarios, such as banking.

Yanli Zhao Guang Yang et al. [16] introduced hybrid Self-Attention Deep Long Short-Term Memory (SA-DLSTM) to predict stock-price movements. SA-DLSTM is a deep learning-based framework that combines multiple models to improve the prediction accuracy. The framework integrates Convolutional Neural Networks (CNNs) for feature extraction, Long Short-Term Memory (LSTM) networks for temporal dependencies, and neural networks for final predictions. The model uses historical stock data and external factors such as market indices and economic indicators to create a comprehensive dataset. After applying self-attention to the input sequences, the attention weights highlight the relevance of each time step, enabling focus on the key features. The weighted representations were processed through LSTM layers to capture temporal relationships and patterns in stock prices. The integration of self-attention and deep LSTM layers improved the capacity of the model to analyze and predict intricate time-based stock price trends. The results show that the framework outperforms standalone models, such as CNN, LSTM, and traditional approaches, in terms of accuracy and F1-score. This framework can handle complex nonlinear relationships in stock data, and is robust to noise. Its modularity allows additional data sources and components. However, it faces challenges in terms of computational complexity when combining self-attention with deep LSTM networks, and requires large datasets to avoid overfitting.

Hong et al. [10] introduced SpectralGPT, a Spectral Remote Sensing Foundation Model, a transformer-based model following generative pretraining principles for spectral remote sensing data that aligns with AI-based stock market prediction. SpectralGPT’s ability to capture sequential patterns, perform multi-target reconstruction, and handle high-dimensional data addresses financial forecasting challenges, including the integration of multiple variables (e.g., historical prices, technical indicators, and external factors). Its pretraining on large datasets parallels financial AI models that leverage historical market data to enhance the predictive accuracy. The processing of spatial, spectral, and temporal data using SpectralGPT suggests its adaptability to financial datasets. The attention mechanisms of the model can focus on key market indicators, similar to the prioritization of spectral features in remote sensing tasks. Spectral decomposition reduces market volatility noise and generative scenario simulation aids risk management. Its ability to handle diverse inputs demonstrates cross-domain applicability. UrbanSAM, designed for urban segmentation, offers insights into financial forecasting through multiscale contextual analysis and cross-attention mechanisms, enabling the alignment of diverse data sources while preserving temporal-spatial fidelity. Its spatiotemporal correlations analyze market influences and its dynamic adaptation suits nonstationary financial time series and risk assessments. UrbanSAM integrates real-time data for short-term market predictions and enables generalization across heterogeneous datasets. Although neither model directly addresses stock market prediction, its methodologies for handling noise, complex data, and sequential dependencies have inspired advancements in financial forecasting.

Etaf Alshawarbeh et al. [17] introduced a hybrid Autoregressive Integrated Moving Average Artificial Neural Network (ARIMA -ANN) for analyzing and predicting high frequency datasets, with a focus on financial time series. The Auto-Regressive Integrated Moving Average (ARIMA) model handles the linear components of the data, whereas an Artificial Neural Network (ANN) addresses nonlinear relationships, thereby creating a complementary framework. This hybrid model overcomes the limitations of the standalone ARIMA, such as its inability to capture complex nonlinear patterns, and the standalone ANN, which struggles to handle seasonality and trends. The methodology was tested on high-frequency financial datasets and achieved significant improvements in accuracy metrics, such as the Mean Squared Error (MSE) and Mean Absolute Percentage Error (MAPE), compared to the individual models. This study highlights the importance of preprocessing steps, such as stationarity checks and data normalization, for hybrid model performance. Although the results were promising, the authors noted challenges in terms of the computational efficiency and the need for careful hyperparameter tuning. This study highlights the potential of hybrid statistical AI models for precise forecasting of high-frequency financial environments.

Tian and Pei [18] implemented the Echo State Network - Sparrow Search Algorithm (SSA-ESN) to predict stock market prices. This method uses Echo State Networks (ESNs) optimized by the Sparrow Search Algorithm (SSA) for hyperparameter optimization. The ESN is a reservoir computing model known for efficiently processing time-series data, owing to its lightweight architecture and dynamic memory capabilities. Traditional ESNs often have suboptimal hyperparameter settings, such as reservoir size and spectral radius, which are addressed by SSA. SSA avoids local optima and searches for large spaces, thereby enabling an effective ESN parameter optimization. This combination helps identify trends in stock prices, with SSA improving the data quality for ESN modeling. The authors showed that the SSA-optimized ESN outperformed standard ESNs and other models (ARIMA and LSTM) in terms of prediction accuracy using metrics such as the Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE). The model demonstrated the ability to capture nonlinear patterns while maintaining the computational efficiency. Despite its advantages, this method faces challenges in terms of the computational complexity and parameter tuning. This study demonstrated the potential of combining evolutionary algorithms with reservoir computing for financial forecasting.

Lu and Xu [19] introduced a Time-series Recurrent Neural Network (TRNN) as a novel architecture to improve the efficiency and accuracy of stock price predictions. The TRNN leverages Long Short-Term Memory (LSTM) units to capture short-term fluctuations and long-term dependencies in financial time-series data. The model uncovers temporal patterns and adapts them to dynamic market conditions by integrating the recurrence mechanisms. Time-series decomposition was employed to handle nonstationary data by dividing them into trend, seasonal, and residual components for the analysis. Optimization techniques were incorporated to reduce the computational overhead while maintaining accuracy. The experimental results show that the TRNN outperforms traditional Recurrent Neural Networks (RNN), LSTMs, and GRUs using metrics such as the Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). However, this model also has several limitations. Its high computational demands make it resource-intensive for real-time trading and less scalable for large datasets. TRNNs are prone to overfitting with poorly generalized training data and require careful regularization. Furthermore, hyperparameter tuning is time consuming, which may hinder its practical implementation. Despite these challenges, this study highlights TRNN’s potential of TRNNs to advance deep learning models for financial forecasting.

5. Taxonomy

The taxonomy of artificial intelligence (AI)-based stock market price trend prediction models provides a structured framework for categorizing methodologies based on their algorithms, data dependencies, and application contexts. This section outlines the classification of these models into distinct categories, and highlights their strengths, limitations, and data dependencies.

Fig. 1 shows a structured taxonomy based on general advancements in AI in financial markets.

Fig. 1Represents the taxonomy of stock market

6. Dataset comparison

Datasets are fundamental to the development of AI-based stock market price trend prediction models, significantly influencing their accuracy, robustness, and generalizability. This section provides a comparative analysis of datasets used in recent studies, focusing on their characteristics, applications, and associated challenges.

7. Comparative analysis

Artificial Intelligence (AI) techniques, including Machine Learning (ML) and Deep Learning (DL), have significantly advanced stock market price trend predictions. This section presents a comparative evaluation of various AI-based methodologies, focusing on their models, datasets, performance metrics, and limitations. The analysis highlights how these approaches address challenges such as market volatility, data noise, and real-time processing while identifying gaps for future research.

8. Gaps and challenges

The application of artificial intelligence (AI) to stock market price trend prediction has demonstrated significant advancements. However, several critical gaps and challenges limit their robustness, generalizability, and practical applicability. These challenges can be categorized into four main areas: data challenges, methodological limitations, interpretability issues, and ethical and regulatory concerns.

9. Future directions

Despite significant advancements in artificial intelligence (AI)-based stock market price trend predictions, critical gaps and challenges persist. These include data-related issues, such as noisy and nonstationary data, methodological limitations, limited interpretability, and ethical concerns. Addressing these challenges requires an interdisciplinary approach combining expertise in finance, machine learning, data science, and ethics. Key research directions for advancing the field toward more reliable, scalable, adaptable, explainable, and ethical AI-driven financial forecasting systems.

10. Conclusions

The application of artificial intelligence (AI) to stock market price trend prediction has significantly advanced financial forecasting by addressing the complexities of volatile and nonlinear financial markets. This review synthesizes insights from recent advancements, showcasing the diversity of methodologies, datasets, and hybrid approaches that have emerged in this field and highlights the transformative potential of AI-driven models in enhancing predictive accuracy, robustness, and scalability.

Hybrid approaches, such as ARIMA-ANN [17] and K-means-LSTM [6], have demonstrated their ability to integrate statistical methods with deep learning techniques, effectively capturing both linear and nonlinear dependencies in the data. GAN-based models, such as MMGAN-HPA [3] and StockGAN [7], show promise in handling noisy datasets by generating high-quality synthetic data, whereas frameworks, such as LSTM-ARO [13], optimize temporal dependencies through advanced hyperparameter tuning. Sentiment analysis models such as Stock Senti WordNet (SSWN) [8] leverage real-time social media data to improve short-term predictions. Frameworks such as HQNN [9], which integrate FinBERT for sentiment analysis, have shown up to an 18 % improvement in prediction accuracy for volatile stocks, emphasizing the importance of incorporating diverse data sources, including historical prices, technical indicators, and social media sentiments. Emerging technologies, such as transformer-based architectures (e.g., SpectralGPT [10]), further enhance the ability to process high-dimensional financial data by capturing sequential patterns and reducing noise. Explainable AI (XAI) techniques, such as SHAP values integrated into HQNN frameworks [9] are paving the way for greater transparency and trust in AI-driven systems.

Despite these advancements, several challenges persist, including data-quality issues, overfitting risks, computational complexity, and ethical concerns. Addressing these gaps requires interdisciplinary collaboration between finance, machine learning, and ethics to develop adaptive learning frameworks, multimodal data integration techniques, and robust ethical guidelines.

In conclusion, AI-based stock market predictions have immense potential to transform financial forecasts. By balancing innovation with interpretability and fairness, future research could enable scalable, transparent, and ethical AI systems to meet the evolving demands of global financial markets.