I have a free tier AWS t2.micro that I use for development purposes. It sits idle most of the time. I figured I would just put it to use, and after some time, I settled on mining Monero, as I can do it with a CPU. My question is this: Are mining pools really necessary? I feel like if you get a payout, the pool splits the payout into a huge amount of users. Is it beneficial to use one or am I better off mining individually?
Mining pools are not necessary for Monero to work. However, they provide a service to small miners, by decreasing the expected variance in income.
Whether or not you use a pool, a miner has about the same long term expected income (modulo pool fee, typically up to 2%, scams, etc). However, long term here can be measured in years if your hash rate is small (see analogy at the end). Therefore, it is quite possible that if you're a small miner, you might not get anything at all for a year. Or you might get the jackpot. For many people, this is unwanted, as they need to pay the electricity bill regularly, so regular income is needed. Since mining pools aggregate their miners' hash rates, they will typically experience less variance.
An analogy: imagine you're betting by flipping coins: You pay 1 monero to play once, and either winning 3 monero if you get heads, or not winning anything if you get tails.: each flip has a 50% change of turning out heads (let's cal, and 50% tails, so the expected long term average is 1 monero per bet: either you lose 1 monero (your fee), or win 2 (3 prize, minus your 1 fee).
Now, if you play just once, you'll either get -1 or 3. A far cry from 1. If you play twice, you can either get -1-1=-2 (-1/bet), -1+3=2 (1/bet), 3-1=2 (1/bet), 3+3=6 (3/bet). The mode is 1, but only 50% chance to get it. If you play 4 times, you can get (per bet) -1, 0, 0, 1, 0.25, 1, 1, 2, 0, 1, 1, 2, 1, 2, 2, 3. The mode is 1, and you see that the expected values start to converge closer to 1. This is the law of large numbers.