I think the question that many decision-makers have to ask themselves, regardless of where they stand on things, is "How would/will we know that it's okay to 'lighten up' on security measures, or is this just the way things are going to be for the rest of our lives?".
No matter how necessary you think it is, or was, there is no disputing that such measures cost wads of money, and incur certain risks and inconveniences. So there are incentives for wanting to eventually go back to the "way things were" prior to the Patriot Act. How would someone know that it was okay to do so? What would the criteria be, for citizens, for legislators, for folks "here" vs "there", for people deeply involved in security culture vs people outside of it?
These are tough questions. I sure wish someone could answer them.