The emergence of prosumers in distribution systems has enabled competitive electricity markets to transition from traditional hierarchical structures to more decentralized models such as peer-to-peer (P2P) and community-based (CB) energy transaction markets. However, the network usage charge (NUC) that prosumers pay to the electric power utility for network services is not adjusted to suit these energy transactions, which causes a reduction in revenue streams of the utility. In this study, we propose an NUC calculation method for P2P and CB transactions to address holistically economic and technical issues in transactive energy markets and distribution system operations, respectively. Based on the Nash bargaining (NB) theory, we formulate an NB problem for P2P and CB transactions to solve the conflicts of interest among prosumers, where the problem is further decomposed into two convex subproblems of social welfare maximization and payment bargaining. We then build the NUC calculation model by coupling the NB model and AC optimal power flow model. We also employ the Shapley value to allocate the NUC to consumers fairly for the NUC model of CB transactions. Finally, numerical studies on IEEE 15-bus and 123-bus distribution systems demonstrate the effectiveness of the proposed NUC calculation method for P2P and CB transactions.