It's the spine-cracking clockwork that does it for me.
That, and a lot of the faces were from that era where people making things (in theory) for children hadn't worked out the difference between "cute" and "Lovecraftian horror."
It's absolutely both of those things, combined with my Mom's ill timed decision to explain to me that those "old toys" were being recycled into "neat new things" only days after reading The Velveteen Rabbit to me. I was very much under the impression that all toys and other inanimate objects were sentient, and was devastated to think of them being destroyed. This fear was only exasperated with Brave Little Toaster coming along a few years later.
Fortunately, by the time Toy Story was released I was already too busy smoking thin black cigarettes and renouncing the Triune God to be concerned.
It was on regular rotation when I was a youngin' growing up on Seasame Street episodes. I never thought this segment was creepy in any way. But... yeeeaaaah nowadays I can see why one might think that.