Puppy Linux was pretty sweet back then, I used it for a Gecko machine for a few years until I got a vastly more powerful 'flat' netbook that arrived after those. It was a pretty nice gadget, though one had to have small and/or flexible fingers or the keyboard would have been a pain.

https://www.digitalreviews.net/reviews/pc/norhtec-xcore-geck...

I've noticed Puppy is still around but I have no idea whether it can still be comparable to Tiny Core.