Are any other people scared of being a worker in their 40's or 50's?

They're scared for the reasons you mentioned. Avoiding bad situations and cirstances is the responsibility of the individual. You seem to be on top of it.