Zwitterionic materials have shown their excellent performance in many biological and chemical applications. Zwitterionic materials possess moieties that own both cationic and anionic groups. The associations among zwitterionic moieties through electrostatic interactions play an important role in properties of zwitterionic materials. However, the relationship between the molecular structures and associations of zwitterionic moieties are still not well understood. This work compared thermal- and salt-responsive behaviors of sulfobetaine and carboxybetaine polymers by examining their rheological properties as a function of temperature and their hydrodynamic sizes as a function of salt concentration. Results showed that carboxybetaine polymers do not exhibit stimuli responses as expected from the antipolyelectrolyte behavior of zwitterionic polymers as observed in sulfobetaine polymers. We studied and compared the associations among zwitterionic moieties in these two zwitterionic polymers using molecular dynamic simulations. Simulation results show that the charge-density difference between cationic and anionic groups determines the associations among zwitterionic moieties, which are responsible for different stimuli responses of carboxybetaine and sulfobetaine polymers.