Do I really need to install a task manager?
I am using ASUS Transformer. I think it uses Ice Cream sandwich, from the UI. I tried many task killers but wonder if they actually work. When I kill tasks, I still see them when I click the "Window selector" 3rd button of the bottom left screen (the 2 boxes). They do appear to take a while to load, so I suppose the tasks were killed but the thumbnails are still there?
Task Killers do generally "work" in that they kill tasks.
They don't "work" in that most apps that you'd want to kill due to running constantly will simply be restarted by the OS, and Android already has appropriate task management built in. A task manager will mostly just drain your battery and waste your time.
See also: How can I stop applications and services from running? and the question eldarerathis links in the comments.
Please read this informative article written in September 2010 for LifeHacker entitled "Android Task Killers Explained: What They Do and Why You Shouldnxe2x80x99t Use Them". In the article it will outline:
-How Task Killers work
-When you should/shouldn't use them
-What you can do as an alternative
Considering the date of your question you it can't be Ice Cream Sandwitch (released on 24th Feburary 2012) but it is instead Honeycomb 3.2
As already suggested in comments by @onik the button you call "Window Selector" display the list of recent opened apps, it does not have any link with apps actually executing on your tablet.
task killer will save battery life on some androids, and makes battery like worse on others. if i were you i would do a test, one day without using task killer and another day using task killer and see if you can find a difference