In the past thirty years, techniques have been developed to cool and trap neutral atoms at ultracold temperatures (<200mK). Cold atomic samples have yielded significant scientific advances in experimental quantum mechanics, precision measurements, and quantum information. Laser cooling, the most common method, is limited to atoms with specific electronic energy structure. Buffer gas cooling is a more widely applicable technique that is not limited by energy level structure. This method thermalizes neutral atoms via elastic collisions with a helium buffer gas maintained at cryogenic temperatures (~100mK). Once the atoms have been cooled they can load them into a magnetic trap. We have successfully engineered the first buffer gas apparatus capable of trapping atoms with magnetic moments as small as 1 Bohr magneton. Further, we have trapped samples of more than 1012 Li, Cu, and Ag atoms at sub-Kelvin temperatures. I will discuss the physical principles involved and the technologies required for buffer gas cooling. I will also discuss how inelastic collisions can change the quantum states of atoms, limiting their lifetime in the trap or preventing trapping altogether.