That is a curious result. I do not know of any reason that ammonia should affect the magnetism of an iron nail. Usually iron nails are not magnetic, but if exposed to a strong enough magnetic field, they can become magnetic. I wonder whether the other nails you tested were also magnetic, but you didn't happen to notice?

Also, when I did the same thing with milk, the nails turned a dark bluish-grey. Why?

Most nails are coated with some other metal, for example many are galvanized with zinc (this means that there is a thin layer of zinc coating the iron to prevent rust). It's possible that the coating on the outside of the nail would have some effect on what's happening here, so it is hard to give an exact answer. But I would guess that what happened is that some proteins from the milk glommed on to the outside of the nail, giving it that blue-gray color. I doubt that there was any actual chemical reaction with the iron or its coating - it is more likely that there is just a coating of proteins and such from the milk on the outside of the nail.