Learning to Infer User Hidden States for Online Sequential Advertising

In online advertising, the optimization of the advertising strategy is crucial. While only knowing click and purchase actions, it is needed to predict user mental states or intents.

A recent study published on suggests using current advances of deep learning techniques to interpret the consumer’s mental states. The probability of these hidden states is estimated by learning from large-scale real-world data, instead of aggregating the consumer’s historical behaviors straightforwardly.

Depending on the user browsing behavior, the customers’ mental state is identified as awareness, interest, or search state. Then, possible state transitions are estimated and the advertising strategy which leads to the biggest reward is chosen. During an experiment in the live ad platform, 9.02 % more revenue was generated with the same budget cost compared with a baseline strategy.

To drive purchase in online advertising, it is of the advertiser’s great interest to optimize the sequential advertising strategy whose performance and interpretability are both important. The lack of interpretability in existing deep reinforcement learning methods makes it not easy to understand, diagnose and further optimize the strategy. In this paper, we propose our Deep Intents Sequential Advertising (DISA) method to address these issues. The key part of interpretability is to understand a consumer’s purchase intent which is, however, unobservable (called hidden states). In this paper, we model this intention as a latent variable and formulate the problem as a Partially Observable Markov Decision Process (POMDP) where the underlying intents are inferred based on the observable behaviors. Large-scale industrial offline and online experiments demonstrate our method’s superior performance over several baselines. The inferred hidden states are analyzed, and the results prove the rationality of our inference.


Comment this news or article