Metastatic tumors (MTs) may show different characteristics of the immune microenvironment from primary tumors (PTs) in non-small cell lung cancer (NSCLC). The heterogeneity of immune markers in metastatic NSCLC and its associated factors has not been well demonstrated. In this study, 64 surgically resected specimens of paired PTs and MTs were obtained from 28 patients with NSCLC. Multiplex immunofluorescence (mIF; panel including programmed death-ligand 1 (PD-L1), Cytokeratin, CD8, and CD68) was performed on whole sections. The heterogeneity of the immune contexture of PD-L1 expression, infiltrating lymphocytes, and immune-to-tumor cell distances was quantified via digital image analysis. In a quantitative comparison of MTs and corresponding PTs, MTs showed higher PD-L1 expression levels, lower density of CD8+ cytotoxic T lymphocytes (CTLs), and longer spatial distance between CTLs and tumor cells. Subgroup analysis, which associated clinical factors, revealed that the heterogeneity of immune markers was more obvious in extrapulmonary, metachronous, and treated MTs, while fewer differences were observed in intrapulmonary, synchronous, and untreated MTs. In particular, MTs showed significantly higher PD-L1 expression and lower lymphocyte infiltration in metastatic NSCLC with EGFR mutations. Prognosis analysis showed that an increased density of CD8+ CTLs in MTs was associated with better overall survival (OS). Therefore, significant discrepancies in PD-L1 expression and lymphocyte infiltration in metastatic NSCLC are most likely associated with temporal heterogeneity with a history of anti-treatment and correlated with EGFR mutations. The detection of immune markers in re-obtained metastatic specimens may be required for immunotherapy prediction in these patients with metastatic NSCLC.