I've started your test and will update it with more results later (unfortunately it's too late for breakfast here), however for the first four thousand files:
Code:
1001 8002 51898
0m02.90s real 0m00.17s user 0m00.41s system
2001 16002 103788
0m07.02s real 0m00.12s user 0m01.09s system
3001 24002 155678
0m13.13s real 0m00.32s user 0m01.49s system
4001 32002 207568
0m19.76s real 0m00.51s user 0m01.72s system
So it seems much slower from the start and it seems to get worse with every additional 1000 files
For me "ls -l" always hangs at the beginning and then outputs everything bascially at once after some time.