Reinforcement learning for bidding strategy optimization in day-ahead energy market
用深度确定性策略梯度算法处理历史电价数据，生成能随时间最大化利润的阶梯式报价曲线，实验表明该方法无需显式预测模型即可捕捉电价时间模式，帮助市场参与者提高利润率。
In day-ahead markets, participants submit bids specifying the amounts of energy they wish to buy or sell and the price they are prepared to pay or receive. However, the dynamic for forming the Market Clearing Price (MCP) dictated by the bidding mechanism is frequently overlooked in the literature on energy market modeling. Forecasting models usually focus on predicting the MCP rather than trying to build the optimal supply and demand curves for a given price scenario. This article develops a data-driven approach for generating optimal offering curves using Deep Deterministic Policy Gradient (DDPG), a reinforcement learning algorithm capable of handling continuous action spaces. Our model processes historical Italian electricity price data to generate stepwise offering curves that maximize profit over time. Numerical experiments demonstrate the effectiveness of our approach, with the agent achieving up to 85% of the normalized reward, i.e. the ratio between actual profit and the maximum possible revenue obtainable if all production capacity were sold at the highest feasible price. These results demonstrate that reinforcement learning can effectively capture complex temporal patterns in electricity price data without requiring explicit forecast models, providing market participants with adaptive bidding strategies that improve profit margins while accounting for production constraints.