Purpose: Time-of-flight joint attenuation and activity positron emission tomography reconstruction requires additional calibration (scale factors) or constraints during or post-reconstruction to produce a quantitative mu-map. In this work, the impact of various initializations of the joint reconstruction was investigated, and the initial average mu-value (IAM) method was introduced such that the forward-projection of the initial mu-map is already very close to that of the reference mu-map, thus reducing/minimizing the offset (scale factor) during the early iterations of the joint reconstruction. Consequently, the accuracy and efficiency of unconstrained joint reconstruction such as time-of-flight maximum likelihood estimation of attenuation and activity (TOF-MLAA) can be improved by the proposed IAM method. Methods: 2D simulations of brain and chest were used to evaluate TOF-MLAA with various initial estimates which include the object filled with water uniformly (conventional initial estimate), bone uniformly, the average mu-value uniformly (IAM magnitude initialization method), and the perfect spatial mu-distribution but with a wrong magnitude (initialization in terms of distribution). 3D GATE simulation was also performed for the chest phantom under a typical clinical scanning condition, and the simulated data were reconstructed with a fully corrected list-mode TOF-MLAA algorithm with various initial estimates. The accuracy of the average mu-values within the brain, chest, and abdomen regions obtained from the MR derived mu-maps was also evaluated using computed tomography mu-maps as the gold-standard. Results: The estimated mu-map with the initialization in terms of magnitude (i.e., average mu-value) was observed to reach the reference more quickly and naturally as compared to all other cases. Both 2D and 3D GATE simulations produced similar results, and it was observed that the proposed IAM approach can produce quantitative mu-map/emission when the corrections for physical effects such as scatter and randoms were included. The average mu-value obtained from MR derived mu-map was accurate within 5% with corrections for bone, fat, and uniform lungs. Conclusions: The proposed IAM-TOF-MLAA can produce quantitative mu-map without any calibration provided that there are sufficient counts in the measured data. For low count data, noise reduction and additional regularization/rescaling techniques need to be applied and investigated. The average mu-value within the object is prior information which can be extracted from MR and patient database, and it is feasible to obtain accurate average mu-value using MR derived mu-map with corrections as demonstrated in this work. (C) 2016 American Association of Physicists in Medicine.