I think that's a fair analysis of current anxieties. But many Polish people I socialise with are genuinely worried about being sent home. Many of whom have made homes and families and good businesses here.
I see nothing wrong in EU people being asked to leave if the final agreement between the UK's elected government and the EU removes their right to live here, just as I see nothing wrong in them being allowed to stay if that's what the agreement permits.
This isn't a matter of right or wrong, but of politics, economics and finance.
I will accept the government's decision and if I am unhappy with it I'll speak out and vote accordingly at election time.
International agreements are not set in stone. They can be ended or altered. Everyone knows that, including the thousands of people who came here after 2004 when an international agreement did change and they were granted the previously unavailable right to live and work in the UK. People who live in a foreign country need to respect the laws of that country, and if those laws are altered to say they have to leave, then so be it. They must accept that change just as they accepted the previous one which allowed them to come in the first place. The EU is composed of safe, first world countries so there are no human rights implications in asking them to leave if that's what our elected government decides.
Even if it happens, which is not certain or even likely, ending the right of EU people to live and work here would hardly be a huge departure from a long held right sanctified and dignified by the passing of time. Those rights have only existed for a short time, and only since 2004 in the case of newer EU member states. The UK does not extend those rights to the vast majority of the world's population, so removing the rights for EU people would simply mean we revert to treating them in the same way as we treat Americans, Australians, Indians, Nigerians and lots of other people in terms of immigration and employment rights. If anything this would be a move towards fairness and away from discriminating in favour of people from a few, overwhelmingly white countries with only about 7% of the world's population.
I'd have no problem with being asked to leave a country if its laws on residence and employment of foreigners changed. I wouldn't dream of expecting the government of a foreign, sovereign nation to base its laws around my needs and desires. Its duty is to its own citizens first and foremost, and then to refugees from dangerous places. There is no obligation to anyone else.