Size doesn't always mean strength. Sure, you probably have weight on a smaller opponent and could pin them, but they'll need smaller openings to get by you. You can compensate for that, just like a shorter person can compensate for a short reach.

Really, there's too many factors when it comes down to a fight, and a lot of them come down to 'how trained are you' and 'how outnumbered are you'.