The multi-directional flow of energy in a multi-microgrid (MMG) system and different dispatching needs of multiple energy sources in time and location hinder the optimal operation coordination between microgrids. We propose an approach to centrally train all the agents to achieve coordinated control through an individual attention mechanism with a deep dense neural network for reinforcement learning. The attention mechanism and novel deep dense neural network allow each agent to attend to the specific information that is most relevant to its reward. When training is complete, the proposed approach can construct decisions to manage multiple energy sources within the MMG system in a fully decentralized manner. Using only local information, the proposed approach can coordinate multiple internal energy allocations within individual microgrids and external multilateral multi-energy interactions among interconnected microgrids to enhance the operational economy and voltage stability. Comparative results demonstrate that the cost achieved by the proposed approach is at most 71.1% lower than that obtained by other multi-agent deep reinforcement learning approaches.