A “pair” means two, but when we speak of a pair of pants, we mean only one article of clothing.
Well, the answer goes back to a time when pants (or pantaloons) were actually two separate pieces of fabric. You had to put them on separately. You had to put one on one leg, tie it around your waist, and then repeat the same for the other piece.
I guess the phrasing was retained even when pants became one article of clothing.
Shirts only came in one piece, so I guess no one was ever inspired to call it a pair of shirts.
Knowing is growing.