The rapid growth of Internet-based applications, such as social media platforms and blogs, has resulted in comments and reviews concerning day-to-day activities. Sentiment analysis is the process of gathering and analyzing people’s opinions, thoughts, and impressions regarding various topics, products, subjects, and services. People’s opinions can be beneficial to corporations, governments, and individuals for collecting information and making decisions based on opinion. However, the sentiment analysis and evaluation procedure face numerous challenges. These challenges create impediments to accurately interpreting sentiments and determining the appropriate sentiment polarity. Sentiment analysis identifies and extracts subjective information from the text using natural language processing and text mining. This article discusses a complete overview of the method for completing this task as well as the applications of sentiment analysis. Then, it evaluates, compares, and investigates the approaches used to gain a comprehensive understanding of their advantages and disadvantages. Finally, the challenges of sentiment analysis are examined in order to define future directions.
Avoid common mistakes on your manuscript.
Sentiment analysis has gained widespread acceptance in recent years, not just among researchers but also among businesses, governments, and organizations (Sánchez-Rada and Iglesias 2019). The growing popularity of the Internet has lifted the web to the rank of the principal source of universal information. Lots of users use various online resources to express their views and opinions. To constantly monitor public opinion and aid decision-making, we must employ user-generated data to analyze it automatically. As a result, sentiment analysis has increased its popularity across research communities in recent years. Sentiment analysis is also called as Opinion analysis or Opinion mining. We have seen a recent growth in the sentiment analysis task. The variuos research works in sentiment analysis (Ligthart et al. 2021) published an overview on Opinion mining in the earlier stage. In (Piryani et al. 2017) discusses the study topic from 2000 to 2015 and provides a framework for computationally processing unstructured data with the primary goal of extracting views and identifying their moods. Several recent surveys (Yousif et al. 2019; Birjali et al. 2021) authors has described the problem of sentiment analysis and suggested potential directions. Soleymani et al. (2017) and Yadav and Vishwakarma (2020) on sentiment classification have been published. Also the topic of detecting opinion spam and fraudulent reviews was investigated. Additionally, In the work of Yue et al. (2019) and Liu et al. (2012) conducted research on the effectiveness of internet reviews. The Authors (Jain et al. 2021b) discuss machine learning applications that incorporate online reviews in sentiment categorization, predictive decision-making, and the detection of false reviews. In the work of Balaji et al. (2021) conducted a thorough examination of the several applications of social media analysis utilizing sophisticated machine learning algorithms. Authors present a brief overview of machine learning algorithms used in social media analysis (Hangya and Farkas 2017).
The growth of social network sites has generated a slew of fields devoted to analyzing these networks and their contents in order to extract necessary information. Sentiment analysis is concerned with deriving the sentiments communicated by a piece of text from its content. Sentiment analysis is a subfield of NLP and that, given long and illustrious public opinion for decision making, there must be multiple early works addressing it. However, it still works going on sentiment analysis develop till the new millennium.
Several real-world applications require sentiment analysis for detailed investigation. for example, product analysis, discover which components or qualities of a product appeal to customers in terms of product quality. In work of Subhashini et al. (2021) presents the results of a comprehensive review of contemporary opinion mining literature. It also covers how to extract text features from opinions with noise or uncertainty, represent knowledge in opinions, and categorize them. Mowlaei et al. (2020) suggest a technique for adaptive aspect-based lexicons for sentiment classification. The authors described two strategies for constructing two dynamic lexicons to aid in the classification of sentiments depending on their aspects: a strategy based on statistics and genetic algorithms. A dynamic lexicon can be automatically updated and provides more precise grading for context-related concepts (Kumar and Uma 2021). To organize all the aspects of the reviews they selected a number of lexicons from several dictionaries. Sentiment analysis is previously being applied in various domains ranging from hotels to airlines and healthcare to the stock market (Zvarevashe and Olugbara 2018). Sentiment analysis has applied to hotel reviews to get a better understanding of customer’s likes and dislikes. In comparison (Valencia et al. 2019), it is used to determine the trends of the stock market and cryptocurrencies based on the market sentiment. The Authors (Ahmad et al. 2019a) analyze tweets relating to various domains and analyze the sentiments of the tweets. The health care domain is seeing a surge in sentiment analysis applications in recent times, customer opinion analyses (Ruffer et al. 2020; Park et al. 2020; Cortis and Davis 2021; Arora et al. 2021), customer satisfaction analyses are few applications in the healthcare sector (Baashar et al. 2020; Miotto et al. 2018). The business sector has always utilized sentiment analysis for its improvement. Sentiment analysis for various applications like reputation management, market research, and competitor analysis, product analysis, customer voice, etc.
Various issues are associated with sentiment analysis and natural language processing, such as individuals informal writing style, sarcasm, irony, and language-specific challenges. There are many words in different languages whose meaning and orientation change depending on the context and domain in which they are employed. Therefore, there are not many tools and resources available for all the languages. Sarcasm and irony are two of the most critical challenges that have recently attracted the attention of researchers. There has been much development in detecting sarcasm and irony in text. There are many challenges in sentiment analysis. In this work we will analyze the variuos challenges, methodologies, applications, and algorithms that are employed in sentiment analysis. We present the task with comparative data analysis shown in tables, flow charts, and graphs that are simple to comprehend. Various description of abbreviation which use in our work shown in Table 1.
To our understanding, existing surveys frequently skip some of the sentiment analysis techniques in favor of machine learning, transformer learning, and lexicon-based approaches. Although this paper covered all the task as well, it varies from earlier research in that it covers the most frequently used techniques. Additionally, other surveys study sentiment analysis from a particular task, various challenges, or concentrate on a specific issue, such as product reviews. This study provides a comprehensive investigation of sentiment analysis by discussing this area from various perspectives since it encompasses numerous research components connected to sentiment analysis, such as problems, applications, tools, and approaches. This work is highly beneficial to scholars and beginners, as it allows them to access a wealth of knowledge about this area in a single paper. The survey important contributions can be summarized as follows
Sentiment analysis has been investigated on several levels: Document Level, Sentence Level, Phrase Level, and Aspect Level. Sentiment analysis in each level such as document, sentence and phrase, aspect level shown in Fig. 1.
Document-level: Document level sentiment analysis is performed on a whole document, and single polarity is given to the whole document. This type of sentiment analysis is not used a lot. It can be used to classify chapters or pages of a book as positive, negative, or neutral. At this level, both supervised and unsupervised learning approaches can be utilized to classify the document (Bhatia et al. 2015). Cross-domain and cross-language sentiment analysis are the two most significant issues in document-level sentiment analysis. (Saunders 2021) Domain-specific sentiment analysis has been shown to achieve remarkable accuracy while staying highly domain-sensitive. In these tasks, the feature vector is a set of words that must be domain-specific and limited.
Sentence level: In this level of analysis, each sentence is analyzed and finding with a corresponding polarity. This is highly useful when a document has a wide range and mix of sentiments associated with it (Yang and Cardie 2014). This classification level is associated with subjective classification (Rao et al. 2018). Each sentence polarity will be determined independently using the same methodologies as the document level but with greater training data and processing resources. The polarity of each sentence may be aggregated to find the sentiment of the document or used individually. Occasionally, document-level sentiment analysis is insufficient for specific uses (Behdenna et al. 2018). In previous work on sentence-level analysis has been devoted to finding subjective sentences. However, more difficult tasks, such as working with conditional sentences or ambiguous statements (Ferrari and Esuli 2019). In these circumstances, sentence-level sentiment analysis is critical.
Phrase level: Sentiment analysis also be performed where opinion words are mined at phrase level, and classification will be done. Each phrase may contain multiple aspects or single aspects. This may be useful product reviews of multiple lines; here, it is observed that a single aspect is expressed in a phrase (Thet et al. 2010). It has been a hot topic of researchers in recent times. While document-level analysis concentrated on categorizing the entire document as subjective, either positively or negatively, sentence-level analysis is more beneficial, as a document contains both positive and negative statements. Word is the most basic unit of language; its polarity is intimately related to the subjectivity of the sentence or document in which it appears. A sentence containing an adjective has a high probability of being a subjective sentence. (Fredriksen-Goldsen and Kim 2017) Additionally, the term chosen for expression represents the demographic characteristics of individuals, such as gender and age, and its desire, social standing, and personality, other psychological and social characteristics (Flek 2020). As a result, term serves as the foundation for text sentiment analysis.
Aspect level: sentiment analysis is performed at the aspect level. Each sentence may contain multiple aspects; therefore, Aspect level sentiment analysis. Primary attention to all the aspects used in the sentence and assigns polarity to all the aspects after which an aggregate sentiment has calculated for the whole sentence (Schouten and Frasincar 2015; Lu et al. 2011).
Data can be collected from the internet via web scraping, social media, news channels, E-commerce websites, Forums, Weblog, some other websites shown in Fig. 2. Data Collection is the first stage in the Sentiment Analysis. Depending on task sentiment analysis of findings, text data can be combined with other types of data like video, audio, location, etc. A few essential sources of data collection are:
Social media: Social data refers to information gathered via social media networks. It demonstrates how consumers interact with the product by accessing, posting, and exchanging over. Academic study on individual, group and behavior uses social media as a dynamic data source. It refers to Internet apps that are web or mobile-based that enable users to create, access, and trade user-generated content.
Forums: Users can use message boards to discuss various topics, exchange opinions and ideas, and solicit assistance via text messages. Forums are an intriguing source for sentiment analysis due to the dynamic nature of user-generated information. Additionally, researchers can undertake sentiment analysis on a specific domain by leveraging forums as a source (Korkontzelos et al. 2016).
Weblog: A short weblog consists of paragraphs conveying a viewpoint, facts, personal diary entries, or links. Together referred to as posts, that are chronologically sorted with the most recent entry appearing first, in the style of a research article (Kumar and Teeja 2012). Blogs an valuable resource for performing sentiment analysis on a variety of entities (Annett and Kondrak 2008).
Electronic Commerce website: Electronic Commerce websites where users can give evaluations and express their opinions about a particular business or organization. In this instance, websites that do not specifically review sites have millions of reviews, such as e-commerce sites that feature product reviews Footnote 1 or professional review sites such as. Footnote 2 In the work of Jain et al. (2019) conducted a descriptive study of the various airline service classifications.
Subjectivity classification This is frequently assumed to be the first stage in sentiment analysis. Subjectivity classification recognizes subjective hints, emotional phrases, and subjective ideas. Tokens like ’hard’, ’amazing’ and ’cheap’ are identified (Kasmuri and Basiron 2017). These indications are used to distinguish objective or subjective text objects. In work of Kasmuri and Basiron (2017) involves determining whether or not there is a particular subject in the given text. Subjectivity classification aims to keep undesirable objective data items out of subsequent processing (Kamal 2013).
Sentiment classification Sentiment categorization is a well-known researched task in sentiment analysis. Polarity determination is one of the subtasks of sentiment classification, and the term “Opinion analysis” is frequently used while referring to Sentiment Analysis. It is a little duty aimed on determining the sentiment of each piece of text. Polarity is traditionally either positive or negative (Wang et al. 2014). In the work of Xia et al. (2015), the opinion-level context is investigated, with intra-opinion and inter-opinion aspects being finely characterized. Neutral is also included in some cases. With a trained classifier, the cross-domain analysis predicts the sentiment of a target domain. Extracting the domain invariant features and where they are distributed is a commonly used approach (Peng et al. 2018). The cross-language analysis is done similarly by training the model on a dataset from a source language and then evaluating it on a dataset from a different language with limited data. The ambiguity of word polarity is one of the obstacles that sentiment analysis must overcome. In the work of Vechtomova (2017) and Singh et al. (2021b) demonstrated that retrieval-based models provide an alternative to Machine Learning based strategies for word polarity detection. Affective computing and sentiment analysis also have tremendous potential as a subsystem technology for other systems (Cambria et al. 2017). They can augment the capabilities of customer relationship management and recommendation systems by enabling the discovery of which features customers particularly enjoy or the omission of items that have received highly unfavourable feedback from the suggestions.
Opinion Spam Detection Spam Detection has become a significant challenge in sentiment analysis because of the rising interest in e-commerce and review platforms. Opinion spams, often known as fraudulent or phone reviews, are well-written comments supporting or criticizing a product for their benefit. Opinion spam detection seeks to identify three distinct characteristics of a phone review: the review’s content, the review’s metadata, and real-world product expertise (Crawford et al. 2015). Machine learning algorithms are frequently used to assess review material in order to detect dishonesty. The star or point ratings, IP address of the user, geolocation of user, and other information are few Metadata used in detecting spam opinions. In many circumstances, though, it is inaccessible for analysis. Real-world experience and knowledge are included in a third way. For example, if a product with bad ratings and reviews is being rated high for a period, that can be put under suspect and analyzed for opinion spam detection.
Implicit Language Detection Sarcasm, irony, and humor are generally referred to as Implicit Languages. These equivocal and ambiguous form is speech is an arduous task to detect, even by humans sometimes. However, this implicit language is an essential aspect of a sentence and can completely flip the meaning and polarity of the sentence. For instance, consider the phrase ’Brilliant, I am fired’. The word Brilliant is very positive, but it describes irony or sarcasm when combined with later parts, i.e., "I am fired" it makes the phrase "I am fired" more negative. Investigating signs such as emoticons, laughter emotions, and extensive punctuation mark utilization are more classic approaches for detecting implicit language (Fang et al. 2020; Filatova 2012).
Aspect Extraction Aspect level sentiment analysis is mainly composed of three steps aspect extraction, polarity classification, and aggregation. The process of aspect-based sentiment analysis starts with the extraction of aspect, one of the key processes as this differentiates usual sentiment analysis. Aspects can be extracted using a predefined set of aspects which should be carefully predefined based on the domain on which it is used. Other approaches are more sophisticated approaches like Frequency-based methods, syntax-based methods, supervised and unsupervised machine learning approaches. It has been seen that in reviews (Kanapala et al. 2019), few words are used more frequently than others, and these most frequent terms are more likely to turn out as aspects; this straightforward method can turn out into quite a powerful approach by fact that a significant number of approaches. This approach has few shortcomings because all frequent nouns do not refer to aspects, terms like ’bucks,’ ’dollars,’ ’rupees,’ etc. Also, aspects that are not mentioned frequently can be missed by this method. A set of rules can be supplemented with a frequency-based approach to overcome these problems, but these manually crafted rules tend to come from parameters that need to be tuned manually, which is a hectic and time-consuming task. Instead of focusing on the Frequency-based approach. Syntax-based approach can be used as this approach covers the flaws of the frequency-based approach of not detecting less frequent aspects (Bai et al. 2020). In this approach, For example, here, ’Awesome’ refers to an adjective referring to the aspect “food” in ’Awesome food.’ For this approach, many annotated data covering all syntactical relations should be collected for training the algorithm.
Sentiment analysis is incredibly significant since it helps businesses understand their consumers sentiment towards their brand. By automatically classifying the emotions behind social media interactions, reviews, and more, organizations can make informed decisions. Sentiment Analysis refers to the methods and strategies that enable firms to examine data about how their customer base feels about a given service or product. To identify the Polarity:? Indicates whether an emotion is good or negative.? Subject: What is the subject of discussion? Who is the holder of the opinion:? A thing or person that conveys the sentiment.
Sentiment Analysis is a process that analyzes natural language utterances automatically, discovers essential claims or opinions, and classifies them according to their emotional attitude.
Three mainly used approaches for Sentiment Analysis include Lexicon Based Approach, Machine Learning Approach, and Hybrid Approach. In addition, researchers are continuously trying to figure out better ways to accomplish the task with better accuracy and lower computational cost. Overview various methods used in Sentiment Analysis as shown in Fig. 4. General Method about the Data collection, Feature selection and Sentiment analysis task are shown in Fig. 2 which understand the overall scenario of sentiment analysis task and overall method workflow.
Lexicons are the collection of tokens where each token is assigned with a predefined score which indicates the neutral, positive and negative nature of the text (Kiritchenko et al. 2014). A score is assigned to tokens based on polarity such as + 1, 0, − 1 for positive, neutral, negative or the score may be assigned based on the intensity of polarity and its values range from [+ 1, − 1] where + 1 represents highly positive, and − 1 represents highly negative. In Lexicon Based Approach, for a given review or text, the aggregation of scores of each token is performed, i.e., positive, negative, neutral scores are summed separately. In the final stage, overall polarity is assigned to the text based on the highest value of individual scores. Thus, the document is first divided into tokens of single words, where-after the polarity of each token is calculated and aggregated in the end.
The lexicon-based technique is extremely feasible for sentiment analysis at the sentence and feature level. Because no training data is required, it might be termed an unsupervised technique. On the other side, the primary disadvantage of this technique is domain dependence, as words can have several meanings and senses, and therefore a positive word in one domain may be negative in another. For instance, given the word “small” and the sentences “The TV screen is too small” and “This camera is extremely small”, the word “small” in the first sentence is negative, as people generally prefer large screens, whereas in the second sentence it is positive, as if the camera is small, it will be easy to carry. This issue can be overcome by developing a domain-specific sentiment lexicon or by adapting an existing vocabulary.
The advantage of the lexicon-based approach is that not require any training data and is considered an unsupervised approach by some experts (Yan-Yan et al. 2010). The main disadvantage with lexicon-based approach is that it is highly domain orientated and words pertaining to one domain cannot be used in another domain (Moreo et al. 2012). For instance, consider the word huge it may be positive or negative based on the domain in which it is being used. In “the queue for the movie was huge” the word may be considered positive whereas, in “there was a huge lag in network” the word can be considered negative. Therefore, the polarity should be assigned to words carefully, considering the domain. There are mainly two approaches used in Lexicon Based Approaches: Corpus Based and Statistical Approach explain below, Comparative Analysis of Lexicon Based Classification Method and its individual Advantage and Disadvantage are shown in Table 3.
The approach employs semantic and syntactic patterns to ascertain the sentence’s emotion. This approach begins with a predefined set of sentiment terms and their orientation and then investigates syntactic or similar patterns to discover sentiment tokens and their orientation in a huge corpus. This is a situation-specific method that requires a significant amount of labeled data to train. However, it aids in resolving the issue of opinion words with context-dependent orientations.
In the work of Park and Kim (2016) used a corpus based method for sentiment analysis. They used linguistic constraints and connectives to find the sentiment of a new token. For instance, tokens on either side of correlative conjunctions like "AND" tend to have the same orientation while words like "OR", but point out opinion change or the tokens on opposite orientations. Although this idea is popularly known as Sentiment Consistency, in practice, this is not that consistent. So, they constructed a graph that contained tokens in vertices and their corresponding word in edges, after which a linear log model was used to identify if two conjoined adjectives were or same or opposite orientation and later clustered into a set of positive or negative words.
The corpus-based approach has the following types of approaches: Statistical Approach and Semantic Approach as explained below.
Statistical Approach The seed opinion words or co-occurrence patterns can be found using statistical approach. The rough idea behind this approach is that if it appears in positive texts more than negative texts, then it is more likely to be positive or vice versa. The key premise of this approach is that if comparable sentiment tokens are frequently observed in the same environment, they will likely have the same orientation. As a result, the orientation of the new token is determined by the frequency with which it appears alongside other tokens detected in a similar context. In Turney and Littman (2003) approach for calculating mutual information can be used to calculate the frequency of co-occurrences of tokens.
A statistical approach is mostly used in several sentiment analysis applications. One such application is detecting manipulated reviews by running a statistical test of randomness popularly known as training test. In work of Hu et al. (2012) expected that reviews written by customers would have random writing styles due to the random backgrounds of customers. They used a book review dataset from amazon.com to confirm their results but, it was found that close to 10.3 percent of products were subjected to online review manipulations.
LSA is another statistical technique for analyzing links between papers and tokens referenced in the documents in order to generate essential patterns connecting to the documents and phrases. In work of Cao et al. (2011) in used LSA to find semantic qualities from reviews to investigate the effect of various features. They engaged program user feedback dataset from the CNETdownload.com website. Their main objective was to find out why few reviews received helpful votes while few reviews helpful votes. They determined various factors which may affect the helpful voting pattern for reviews.
Semantic Approach In this approach, the similarity score is calculated between tokens that are used for Sentiment Analysis. Wordnet is commonly used for this task. Antonyms and synonyms can be easily found using this approach as similar words have a positive score or higher value. In Maks and Vossen (2012) proposed that semantic approach can be used in various applications to build a lexicon model that can be used to describe adjectives, verbs, and nouns to use in Sentiment Analysis. They described, the in-depth description of subjectivity relations among the characters in a statement conveying distinct attitudes for each character. subjectivity tagged with the knowledge relating to both identity and orientation of attitude holder. In work of Bordes et al. (2014), Bhaskar et al. (2015), Rao and Ravichandran (2009) worked on the WordNet dataset in their work. They determined that the viewer’s subjectivity and the actor’s subjectivity might be distinguished in some instances (Hershcovich and Donatelli 2021).
Dictionary based approach consists of a list of predefined set opinion words collected manually (Chetviorkin and Loukachevitch 2012; Kaity and Balakrishnan 2020). The primary assumption behind this approach is that synonyms have the same polarity as the base word, while antonyms have opposite polarity. Large corpora like thesaurus or wordnet are looked upon for antonyms and synonyms, after which it is appended to a group or seed list prepared earlier. In the first stage, initial set of words are collected manually with their orientation. Later the list is expanded by looking at the antonyms and synonyms in the available lexical resources (Singh et al. 2017; Ho et al. 2014). Then the words are iteratively added to the list, and list is expanded. Manual evaluation or correction may be done in the last stage to ensure the quality of it. Stefano and Andrea created SentiWordNet three-way in Baccianella et al. (2010) with the help of automatic annotations of WordNet \(3's\) synsets. Another famous resource thesaurus was created based on online dictionaries. In the work of Park and Kim (2016) suggested a rule-based strategy for labelling sentiment sentences and words in contextual advertising using a dictionary-based approach. This approach is feasible only for small dictionary size. Another disadvantage of all lexicon-based approaches (Hajek et al. 2020), including the dictionary-based approach, is finding opinion words specific for each domain as the polarity may vary. General Procedure step in Lexicon Unsupervised learning category shown in Fig. 6. Summary Analysis of Lexicon Based Classification Method and its Advantage and Disadvantage shown in Table 3 and Summary Analysis of Clustering Method and its Advantage and Disadvantage shown in Table 2.
NB technique is utilized for both categorization and training. NB is a Bayesian classification approach based on the theorem of bayes. NB is a probabilistic classifier that uses Bayes theorem to predict the probability of a given set of features as part of any particular label. The conditional probability that event A occurs given the individual probabilities of A and B and conditional probability of occurrence of event B. Here it is assumed that features are not dependent. BoW model may be used for feature extraction. Generally, NB is applied when the training data size is small. NB classified as positive 10% more accurately than negative classification. This led to a decrease in average accuracy when it was taken. In the work of Kang et al. (2012) solved this problem using an improved version of the NB classifier. They tested this model on to restaurant review dataset. In work of Tripathy et al. (2015) used machine learning for the classification of reviews. They proposed a NB model along with a SVM model (Hajek et al. 2020; Bordes et al. 2014). They used a movie review dataset for training and testing the models. Two thousand reviews were trained after pre-processing and vectorization of the training dataset. Count Vectorizer and TF-IDF were used before training the machine learning model. NB model proposed in Tripathy et al. (2015) gave an accuracy of 89.05 percent in a K-fold Cross-validation. The performance was better when compared to other models using the probabilistic NB algorithm (Calders and Verwer 2010).
SVM approach, which uses hyper-planes, is used to analyze data and define decision limits in this technique. SVM are a type of non-probabilistic supervised learning technique that is frequently used for classification tasks. SVM primary objective is to determine the hyperplane that best separates the data into distinct classes. As a result, SVM seeks out the hyperplane with the highest feasible margin. In work of Li and Li (2013) used Support Vector Machines for sentiment polarity Classifier. Classifying reviews based on their quality is one of the many purposes for which SVM are utilized. Chen and Tseng (2011) used two multiple class SVM based approaches. First being One-vs- all SVM and Multiclass SVM to classify reviews. Second, a method was proposed to evaluate the quality of the product review dataset quality by considering it as a classification problem. In work of Dave et al. (2003) worked on MP3 reviews and digital camera reviews. Borg and Boldt (2020) used Linear SVM and VADER to predict customer reviews sentiment. The review belonged to Huge Swedish Telecom Corporation. The dataset was huge and consisted of 168, 010 emails for training. They used Swedish sentiment Lexicon and VADER sentiment for initial labeling. Their (Li et al. 2019a) linear SVM model performed marvelously with an F1 score of 83.4 percent and a mean AUC of 0.896. Furthermore, their model highlighted a pattern that was predicted in Email conversations, using which sentiment of unseen Email was being predicted. In work of Xia et al. (2020) urged that subjectivity of opinion and credibility of expresser should be considered, unlike regular Binary Classification Problem. A framework (Wu et al. 2020) was proposed to summarize opinions on microblogs. They found and retrieved the topics mentioned in the opinions related to users inquiries and then categorized the opinions using SVM. Ali et al. (2020) also worked on Twitter tweet data for the experiment. They found it to be beneficial to aggregate the opinions for microblogs.
A machine learning technique known as logistic regression works by multiplying an input value by a weight value. It is a classifier that learns which input properties are most helpful in identifying positive and negative classes. Logistic regression is a probabilistic regression analysis used for classification tasks. For binary classification applications, logistic regression is commonly deployed. When there are multiple explanatory variables, logistic regression calculates the ratio of odds. Logistic regression uses Maximum-likelihood to calculate best parameters. The independent variables may belong to any category i.e., Continuous, Discrete (ordinal and nominal). LR model (Hamdan et al. 2015) that the dependent variable is binary, and there is little or no multicollinearity between the predicting variables.
DT Classifier is a supervised learning technique where a tree is built using the training example to classify the polarity of the text. DT uses a condition to divide data into parts recursively. RF are used frequently than DT which combines multiple DT to avoid overfitting and improve accuracy. DT may be built using several algorithms like CART, ID3, C5.0, C4.5 (Revathy and Lawrance 2017; Hssina et al. 2014; Singh and Gupta 2014; Patel and Prajapati 2018). These are used the identify the best fitting attribute which needs to be placed in the root (Gower 1966; Revathy and Lawrance 2017; Patil et al. 2012). Yan-Yan et al. (2010)using a graph-based strategy, They proposed a propagation strategy for integrating sentence-level and sentence-level features. These two phrase characteristics are referred to as inter and intra document verification. They tried to argue that determining the sentiment classification of a review sentence entails more than simply examining the statement’s components. They investigated the camera domain and compared their results to those obtained using SVM and NB Classifiers. In the work of Jain et al. (2021a) tagged data that can be used to distinguish between genuine and fraudulent reviews. Additionally, we used two distinct datasets to test various machine learning techniques for categorization (Yelp hotel review dataset, Yelp restaurant review dataset).
Conditional Exponential Classifiers: Conditional exponential classifiers encode labelled feature sets as vectors or arrays of integers. This vector is then used to compute feature weights, which can be used to select the most likely label for the feature set. Entropy is a measure of unpredictability. The Entropy is maximum for uniformly distributed data. The input data consists of texts and ratings from 1-5 polarity assigned to it. Most popularly used algorithms include SVM, NB, ME (Khairnar and Kinikar 2013; Kaufmann 2012) used ME Classifier to detect parallel sentences in any two-language pair, which have less training data. The other models used either required a massive amount of training dataset or used a language-specific technique (Bergsma et al. 2012), but their model showed improved results could be produced using any pair of languages. This will enable the establishment of parallel corpora for various languages.
KNN algorithm is not extensively used in sentiment analysis but has shown to produce good results when trained carefully. It operates on the fact that the classification of a test sample will be similar to nearby neighbours. The K value may be selected on any hyper-parameter tuning algorithms like Grid search or Randomized search cross validation. The polarity may be hard voted based on K nearest neighbors values, or soft addition may be done to find overall polarity.
In this case, where the training dataset contains both labelled and unlabelled data, semi-supervised learning appears to be a viable option (Zhu and Goldberg 2009). It is motivated that while gathering unlabelled data is relatively easy in many real-world applications, such as collecting articles from various blogs, labelling is expensive or labour-consuming because labelling the training dataset is typically performed by humans. Ortigosa-Hernández et al. (2012) introduced in the work of a real-world situation in which the user attitude is defined by three distinct (but related) target variables: subjectivity, sentiment solarity, and will to influence. In the work of Janjua et al. (2021) framework for semi-supervised machine learning that combines pre-processing and classification algorithms for unlabelled datasets.
Sentiment analysis has many applications, ranging from analyzing customer opinion, analyzing patient mental health status based on posts done on social media. Furthermore, technological advances such as Blockchain, IoT, Cloud Computing, and Big Data have broadened the range of applications for Sentiment Analysis, allowing it to be used in practically any discipline. Few most frequently used application in sentiment analysis shown in Fig. 8. A few significant domains and industries where Sentiment Analysis is applied are described below:
Sentiment analysis in the field of business intelligence offers several benefits. Additionally, firms can utilize sentiment analysis data to improve products, investigate client feedback and develop an innovative marketing strategy. The most typical use of sentiment analysis in the field of business intelligence is analyzing customers impressions of services or products. These studies, however, are not limited to product producers; consumers may use them to review items and make more informed decisions. Sentiment analysis in business intelligence has various benefits. For example, Businesses can use the results of Sentiment Analysis to make product enhancements, examine consumer feedback, or develop a new marketing plan (Han et al. 2019). Sentiment analysis is most frequently used in business intelligence to examine customers perceptions of products or services. However, these analyses are not limited to product producers; consumers may also use them to compare items and make a more informed choice. For eight years, (Bose et al. 2020) service food reviews on amazon.com. Emotion lexicon, which classifies them into eight different emotions and two moods (positive and negative). They found that sentiment analysis may be used to identify customer behaviours and hazards and increase customer satisfaction.
As the e-commerce business is burgeoning, so is the number of products sold and reviews given from the customers. Sentiment analysis one them will help customers choose better products (Paré 2003). Phrase level or aspect level (Schouten and Frasincar 2015) sentiment analysis performed on product reviews. Sentiment analysis can determine what the customer thinks about its latest product after launching or examining comments and reviews. Keywords for a specific product feature (food, service, cleanliness) can be chosen, and a sentiment analysis framework (Mackey et al. 2015) can be trained to identify and analyze only the necessary information.
Market research is perhaps the most common sentiment analysis application, besides brand image monitoring and consumer opinion investigation. The purpose of sentiment analysis is to determine who is emerging among competitors and how marketing campaigns compare. It can be utilized to acquire a complete picture of a brand’s and its competitors consumer base from the ground up. Sentiment analysis may collect data from several platforms Twitter, Facebook, blogs, deliver tangible results, and overcome difficulties in business intelligence.
This is one of the industries where sentiment analysis is being utilized in recent times. Data can be collected from various sources like surveys, Twitter (Carvalho and Plastino 2021), blogs, news articles, reviews, etc. This data can then be analyzed for various use cases, one of them being an evaluation of standards and analysis of new updates in the medical field. Domain experts are researching actively to find more uses of sentiment analysis and other NLP applications (Ebadi et al. 2021). This application helps healthcare service providers collect and evaluate patient moods, epidemics, adverse drug reactions, and diseases to improve healthcare services. In work of Jiménez-Zafra et al. (2019) pointed out the difficulties in applying sentiment analysis in health care because of the specific and unique terminologies used in the domain. In work of Clark et al. (2018) used Twitter tweets concerning patient’s experiences as an add-on to analyze public health. Over a year, they generated roughly five million breast cancer-related tweets using Twitter’s Streaming API. After pre-processing, the tweets were classified with a standard LR classifier and a CNN model. Positive treatment experiences, rallying support, and expanding public awareness were all linked. In conclusion, applying sentiment analysis to analyze patient-generated data on social media can help determine patients’ needs and views.
The application of sentiment analysis in diverse markets is brand monitoring and reputation management. Evaluating how customers view their brand, product, or service is beneficial to fashion companies, marketing agencies, IT companies, hotel chains, media channels, and other businesses. Sentiment analysis tool adds more variety and intelligence to the brand’s and their products portrayal. It enables businesses to track how their customers perceive their brands and highlight the precise data about their attitudes. Look for trends and changes, and pay attention to influencers presentations. Altogether, sentiment analysis can be utilized in automating the media surveillance system as well as the alarm system that goes with it. Keep track of the brand’s discussions and ratings on various social media platforms.
Sentiment analysis is extensively used in the domain of Entertainment. Reviews of movie, shows, and short films may be analyzed to determine the viewer’s response (Kumar et al. 2019). This not only helps viewers make a better choice but also helps good contents gain popularity. Sentence level (Lin and He 2009) Sentiment Analysis has commonly used in this domain to determine the overall sentiment of the reviews given accurately. The travel industry has sought to improve client experiences by developing machine learning and online consumer recommendation systems based on intelligent, data-driven decision-making techniques (Jain et al. 2021f) also discussed categorizing consumer decisions as positive or negative based on online reviews provided by the valuable consumer(Jain et al. 2021e).
Sentimental analysis on reviews on hotels and restaurants can help customers choose better and also help the owners improve. Aspect-based sentiment analysis done on hotels and restaurants will help identify the aspect with the most positive reviews and negative reviews, on which Hotels can work and make it better. (Sann and Lai 2020; Al-Smadi et al. 2018) According to sentiment analysis, this is one of the most attractive industries. Sentimental analysis on reviews on hotels and restaurants can help customers choose better and also help the owners improve (Zhao et al. 2019). ABSA (Akhtar et al. 2017) done on hotels and restaurants will help identify the aspect with the most positive reviews and negative reviews, on which hotels can work and make it better. The service providers profit the most since they may extract the aspect that receives the most negative feedback and improve on it.
Aspect-based sentiment analysis can help businesses make the most use of the massive amounts of data they create. The aspect-based method will enable companies to extract the most important aspects of client feedback and service.
One of the applications of sentiment analysis is stock price prediction. It can be done by analyzing all the news about the stock market and predicting the stock price trends. Data can be collected from various sources like Twitter, news articles, blogs, etc. Sentence level sentiment analysis can be done on these texts, after which the overall polarity of texts will be decided of news of a particular company. In work of Xing et al. (2018) used to determine whether the trend will be rising or decreasing. Positive news tended to lead to an upward trend, whereas negative news tended to lead to a downward trend. Bitcoin and other digital cryptocurrencies relate to a novel technology known as Blockchain. Participants inside the blockchain network verify the digital transactions using peer to peer consensus methods. However, investigations which apply Sentiment Analysis towards the area of blockchain technology are still infrequent, and those that do exist, such as work in Kraaijeveld and De Smedt (2020), have employed sentiment analysis to anticipate the value of digital cryptocurrencies. In the work of Rognone et al. (2020) investigated the influence of news sentiment on cryptocurrencies like bitcoin and other standard currencies volatility, volume, and returns.
Take all user feedback from the call centres, emails, surveys, chats, and web and combine and assess it. Sentiment analysis will allow categorizing and organizing data in order to detect trends and reoccurring issues and worries. Sentiment analysis may aid in identifying an appropriate customer group and subsequent value proposition development, both of which are essential components of a successful business operation. On the other hand, to stay updated and maintain the product in demand, it must have the finger on the pulse of its customers.
Sentiment analysis of social data will monitor client sentiment 24 hours a day, seven days a week, in real-time when anything unpleasant starts to circulate, which can rapidly reply and bolster image when getting favourable mentions. That also obtains consistent, reliable information on clients, which can track progress from season to season for the decision-making process. Because individuals provide their comments without being asked, social media posts frequently present some of the most honest points of view regarding products, services, and enterprises. They are obliged to express their feelings to the rest of the world.
Sentiment analysis comes with various challenges ranging from computational cost to informal writing and the presence of variations in languages. We look at the sentiment analysis challenges that occur more frequently with certain types of sentiment structure, as shown in Table 7. Few significant challenges faced in sentiment analysis are:
Structured sentiments are found in formal sentiment reviews, they are more focused on formal problems such as books or research. Because the authors are professionals, they are capable of writing thoughts or observations concerning scientific or factual concerns.
Semi-Structured Sentiments fall between structured and unstructured sentiments. These require an awareness of numerous review-related concerns. This style, which is dependent on benefits and drawbacks, is listed separately by the authors, and the pros and cons sections are typically comprised of brief sentences (Birjali et al. 2021; Hussein 2018; Ebrahimi et al. 2017; Mohammad 2017).
Unstructured Sentiment is an informal and free-flowing writing type in which the writer is not constrained by any rules (Mukherjee et al. 2013). The text may comprise multiple sentences, each of which could potentially include both pros and cons. For example, unstructured reviews offer more opinion information than their formal counterparts (Levashina et al. 2014). A feature explicitly stated: If a feature occurs in a review sentence’s segment/chunk, the feature is referred to as an explicit feature of the product. For instance, in the segment, the image is marvelous. The image is an explicit feature. If a feature f is not explicitly mentioned in the review section but is implied, it is referred to as an implicit feature of the product (Liu et al. 2010; Elith et al. 2011). For instance, in the section, it is extremely pricey, and expensive is a feature sign. In light of the critical nature of sentiment analysis, this study examines the relationship between respondents perspective structures and sentiment analysis issues.
Table 7 Study of various challenges in sentiment analysis as per structure of dataThe majority of sentiment analysis in the modern day is data-driven machine learning models adapting a sentiment analysis algorithm developed for product evaluations to evaluate microblog postings is an unanswered question. Additionally, how to deal with ambiguous situations and irony are key difficulties in sentiment analysis. For instance, a sarcastic remark about an object is intended to communicate a negative sentiment; yet, conventional sentiment analysis algorithms frequently miss this meaning. Numerous methods have been proposed (Castro et al. 2019; Medhat et al. 2014) for detecting sarcasm in language. However, the problem is far from resolved, as comedy is very culturally particular, and it is challenging for a machine to understand unique(and frequently fairly detailed) cultural allusions. In the work of Poria et al. (2018a) suggest by incorporating vocal and facial expressions into multimodal sentiment analysis; This can improve its success rate in identifying sarcastic comments. Furthermore, individuals express sentiment for social reasons unrelated to their fundamental dispositions. For instance, a person may transmit positive or negative thoughts to adhere to a specific topic A norm or express and define one’s identity. Finally, machine-based sentiment analysis is confined to outward expressions of sentiment, and conclusive information about an individual expressed ideas is lacking.
Sentiment analysis is applicable to different types of data, each of which presents particular challenges. Sentiment analysis of human to machine and human to human interactions requires very similar datasets to those used for emotion recognition. As a result, it has the same limitations in terms of size and unreliable ground truth. In the work of McDuff et al. (2014) have illustrated how webcams may be used to collect a large number of emotional reactions, including sentiment. While this degrades the audiovisual capture quality, it achieves a scale that is not conceivable in the laboratory. Additionally, there is the issue of labeling confidential laboratory data, which prohibits those permitted to examine the data from performing the time-consuming operation of labeling. As a result, they are restricted in terms of the amount of data they can collect in the laboratory and our ability to label huge volumes of data. There are several methods for assessing feelings, but word embedding algorithms such as word2vec and GloVe turn words into meaningful vectors. These methods, on the other hand, ignore the word’s sentiment information (Wankhade et al. 2021).
Multimedia information on websites is the second source of multi-modal sentiment data. Social media provides us with a wealth of data that helps us to scale. The issue is that the data acquired vary in terms of quality and context, and the data is limited to specific populations that are more prevalent on the internet. However, because the data is publicly available, crowd sourcing may be utilized to categorize it easily. According to the available data on MSA, people are more prone to communicate positive or negative ideas online, resulting in a scarcity of neutral opinions represented in all MSA studies evaluated.
Sarcasm People tend to use sarcasm when they do not meet their expectations. It is very tough for machines to pick up sarcasm as many factors affect sarcasm, such as tone, situation, background information, etc. Sarcasm is a satirical remark that may look like praising but in reality. Sarcasm is used by people to criticize. Sarcasm is a type of sentiment in which people express implicit information, usually the polar opposite of the message content, in order to emotionally hurt someone or mock something. Sarcasm detection in text mining is one of the most challenging tasks in NLP, but it has lately become an interesting research subject due to its usefulness in enhancing social media sentiment analysis (Eke et al. 2020).
Informal style of writing Informal style of writing is the biggest challenge to all NLP tasks, including sentiment analysis. People are very casual about writing reviews or texts; they tend to use acronyms, emojis, shortcuts in their text which is very hard to pick up. Acronyms can be handled if they are universal. There are a lot of regional acronyms Footnote 14 which change and grow day by day.
Grammatical errors Grammatical errors are very common in informal texts and can be handled, but only to some extent; spelling errors can also be corrected limited. It is very difficult to burgeoning the spelling mistake of users uniquely every time. The accuracy of sentiment analysis and NLP tasks may be improved if these errors can be handled and corrected.
Computational cost To get better accuracy, we need to increase the training data size and complicate the model, which will exponentially increase the computational cost of the model for training; high-end GPU may be required to train a model with a huge corpus. Models like SVM, NB are not computationally costly, but neural networks and attention models have shown that they are computationally costly.
Availability of data As NLP and sentiment analysis is a recently boomed technology, the Availability of data may also be a challenge in some cases. Although data is available in Twitter for sentiment analysis, high-quality training data is challenging for supervised learning algorithms. Training data for ABSA is challenging to find online therefore needs to be prepared manually. The training data of one domain may not be applicable and valuable to other domains. For instance, a model trained on a hotel review dataset is not helpful in predicting sentiments of a stock or mutual fund dataset and vice versa.
Adaptations of language Languages change as they move to different regions and places; although the base language remains the same, many factors influence language, such as language prominence, pronunciation, literacy rate, etc. For instance, consider English language, which is widely spoken worldwide, but it is seen that many English varieties are spoken worldwide based on the regions like Indian, American, British, etc. Lots of words are used differently depending upon the region there are used. For instance, consider the word “thong” which means flip-flops or slippers in Australia but means undergarments in the UK. Similarly, different spellings for the same word, such as “color” and “colour,” mean the same but are spelled differently in different regions. This will create duplicates and may affect the accuracy and computational cost of the model. Language barrier is the hardest of the challenges to NLP. There are thousands of languages spoken worldwide, although NLP techniques are hardly available to 5-10 languages, and resources are widely available for English.
Phrases containing degree adverbs and intensifiers Adverbs such as slightly, barely, and moderately are used to quantify the sentiments. For instance, consider review r1= “The food is barely good” and r2= “the food is really good”. r1 is considered neutral or slightly positive, whereas r2 is considered to be highly positive. The adverbs ’barely’ and ’really’ decide the extent of positiveness and the word ’good’. Similarly, intensifiers also quantify the sentiment of the sentences. Intensifiers like very, too are used to increase the positiveness or negativeness of the token. For instance, “too good” is considered to be more positive than “good.” Intensifiers and degree adverbs impose a challenge on aggregating the sentiment values and comparing two sentences of the same sentiment rather than differentiating between two sentences of opposite polarity.
The theoretical challenges employ a variety of approaches to enhance performance when answering the particular sentiment challenges (Hunter et al. 2012). The theoretical kind makes extensive PoS tagging and lexicon-based approaches (Taboada et al. 2011). The second approach is the BoW technique (El-Din 2016). Finally, there is the ME approach. However, the most frequently used technique is the N-gram technique, which is based on phrases and expressions when it comes to technical sentiment challenges (Wilson et al. 2009). As well as the method that is used the least is the lexicon-based approach.
Mixed Code Data Code-mixing is the employment of vocabulary and grammar from different languages in same sentence (Pravalika et al. 2017; Poria et al. 2020). Code Mixing is a linguistic phenomenon that can occur in a multilingual situation where speakers speak multiple languages. This phenomenon is becoming increasingly common as communication between groups of people who speak different languages grows. Code-Mixing: A review of Facebook posts created by Hindi-English users revealed a high code-mixing level in the posts. The problems in the Hindi-English code-mixed text were reported using a PoS tag annotated corpus. (Vijay et al. 2018) a system that can detect the language of the words, normalize them to their standard forms, assign their PoS tag, and split them into chunks to handle the problem of shallow parsing of Hindi-English code-mixed social media content. It’s frequent in multilingual societies and presents considerable difficulty to NLP tasks like sentiment analysis. The lack of a formal grammar for code-mixed phrases makes it challenging to identify compositional semantics, which is critical for conducting sentiment analysis using rule-based and machine learning-based techniques. Furthermore, because mixing is up to the individual, there are no predetermined mixing guidelines, which is one of the significant drawbacks (Chatterjere et al. 2020). As a result, in order to conduct sentiment analysis on code-mixed data, new language models must be developed. In work of Chatterjere et al. (2020) and Singh et al. (2018) language modeling challenge for code-mixed Hinglish text was investigated. However, despite the fact that Code Mixing is a significant concern, few research has addressed it as thoroughly as the study of Lal et al. (2019) (English-Hindi) code-mixed data for sentiment analysis, the authors presented a hybrid architecture.
This article discussed sentiment analysis and associated techniques. The primary objective of this work is to investigate and complete classification methods with their advantage and disadvantages in sentiment analysis. To begin, several levels of sentiment analysis were discussed, followed by a quick overview of necessary procedures such as data collection and feature selection. Next, methods of sentiment categorization systems were classified and compared in terms of their advantages and disadvantages. Due to their simplicity and excellent accuracy, supervised machine learning methods are often the widely utilized technique in this discipline. Classification using NB and SVM algorithms are commonly used as benchmarks against which newly proposed approaches can be compared. Several of the most common application areas are discussed then the survey examines the significance and consequences of sentiment analysis challenges in sentiment evaluation. The comparison investigates the relationship between the structure of sentiment reviews and the difficulties associated with sentiment analysis. This comparison reveals domain dependence, which is essential for identifying sentiment issues. The future work will consist of continuously expanding the comparison area with additional findings. The subsequent challenges illustrate that sentiment analysis is still a relatively unexplored subject of study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Wankhade, M., Rao, A.C.S. & Kulkarni, C. A survey on sentiment analysis methods, applications, and challenges. Artif Intell Rev 55, 5731–5780 (2022). https://doi.org/10.1007/s10462-022-10144-1
Anyone you share the following link with will be able to read this content:
Get shareable link
Sorry, a shareable link is not currently available for this article.
Copy to clipboard
Provided by the Springer Nature SharedIt content-sharing initiative
