Most progressive agencies allow the carry of a backup gun, and many encourage it. Why not? If you can be trusted with one gun, you can certainly be trusted with two. Plus, there are many documented cases of backup guns saving officers lives.
You can easily and comfortably carry a backup gun in an inexpensive holster affixed to the straps of your ballistic vest. Replace the second button of your shirt with Velcro tabs to make access easy and fast.
Important: Make sure your backup gun has a trigger-pull weight at least as heavy as that on your duty gun. If you have to use it for real, you'll be under a great deal of stress, and a trigger pull lighter than you are used to will very likely result in an accidental discharge. If that happens, you or innocent bystanders are quite likely to be shot.
Many small autos and revolvers make ideal backup guns, but as with your duty gun (or any gun used for self defense), stick to calibers of .38 or 9mm or greater. That's because there's perhaps one thing that all firearms trainers agree on: Lesser calibers are too ineffective for personal defense use. This is why no law enforcement agency in the U.S. (to my knowledge) issues primary duty guns in calibers less than .38/9mm.