An analytic method is proposed to compute the price-reserve offer curve at the consumer level in hierarchical direct load control. The convexification of the consumer reserve provision is examined, and the analytic expression of the optimal solution within each critical region is derived. Then, based on multi-parametric programming, a combinatorial enumeration method in conjunction with efficient reduction and pruning strategy is proposed to compute the optimal response of consumers in the whole price space. Numerical tests along with an application example in the bi-level aggregator pricing problem demonstrate the merit of this method.