Applications of nanofiltration membrane processes to drinking water treatment are increasing. A typical objective is to control natural organic matter (NOM) to reduce disinfection by-product formation. Major considerations include NOM rejection and flux decline during filtration. The objective of this study was to investigate the influence of interactions between natural nanoparticles (silica colloids) and NOM on membrane fouling, particle deposition, and solute (NOM and salt) selectivity. A reduction in NOM selectivity by polyamide NF membrane was observed when the concentration of colloids was high and/or when a thick particle cake formed on the membrane. Moreover, the presence of NOM in the cake layer matrix substantially attenuated the enhancement of salt concentration polarization and increased its retention. The transport processes of both NOM and salt were successfully simulated using a multi-layer hydrodynamic solute transport model. Properties of the first deposited colloidal cake layer were critical in determining permeate quality.