If the pending transactions fit in a block, there's not much incentive to not include them -- it doesn't cost much, so miners can include them all in order to maximise fee revenue. But when there are more pending transactions than fit in a block, miners will sort by fee and include the biggest fees and reject the lowest fees. By growing blocksize, you avoid the situation where there are more pending transactions than can fit in a block.
Miners will likely omit transactions that offer a fee that is lower than their cost of mining (at least eventually). This sacrifices some small amount of revenue on the current block, but should lead to higher revenue over the long term (assuming their cost levels make bitcoin an attractive transaction system).
The cost of mining a single transaction is approximately the electricity cost to mine a block divided by the number of transactions in the block.
Say it is $0.15. A miner can make extra revenue at ~0 cost by filling a block with $0.10 offered fee transactions, but if there are many of them, they may choose not to, from the belief that enough of them will turn into $0.16 offered fees in the future (the cost is clearly ~0 in the short term, it's harder to say what it costs them in the long term to propagate the impression that low fee offers will still eventually clear).
However, that would only become a problem after centralization, correct? Otherwise, it would require a concerted effort by most/all miners to take the same approach, rather than competing for staying power etc.
