Boosted by a strong solar power market, the electricity grid is exposed to risk under an increasing share of fluctuant solar power. To increase the stability of the electricity grid, an accurate solar power forecast is needed to evaluate such fluctuations. In terms of forecast, solar irradiance is the key factor of solar power generation, which is affected by atmospheric conditions, including surface meteorological variables and column integrated variables. These variables involve multiple numerical time-series and images. However, few studies have focused on the processing method of multiple data types in an inter-hour direct normal irradiance (DNI) forecast. In this study, a framework for predicting the DNI for a 10-min time horizon was developed, which included the nondimensionalization of multiple data types and time-series, development of a forecast model, and transformation of the outputs. Several atmospheric variables were considered in the forecast framework, including the historical DNI, wind speed and direction, relative humidity time-series, and ground-based cloud images. Experiments were conducted to evaluate the performance of the forecast framework. The experimental results demonstrate that the proposed method performs well with a normalized mean bias error of 0.41% and a normalized root mean square error (nRMSE) of 20.53%, and outperforms the persistent model with an improvement of 34% in the nRMSE.