I have an etiquette question.

As many of you know my husband and I are frequent flyers. I am also a nervous flyer, so I have to sit next to my husband. Mostly to grab his arm when the turbulence hits.

My husband likes the aisle seat. So, I am often in the middle seat of a three-seat aisle.  I have recently noticed that the person who takes the window seat generally takes both arm rests, the one near the window and the one that adjoins my seat.

On our last flight, I got up so that the person who had the window seat could take their seat. Before I even sat down, he was comfortably seated in his window seat with his arms on the window arm rest and my arm rest. I said nothing, pulled out my kindle to read, and kept my arm off that arm rest throughout the flight. Luckily it was a very smooth flight-we were on the runway before I even knew it-so my husband's arms went unscathed.

When I mentioned the arm rest issue to my husband after the flight, his opinion was that I should have said something to the person in the window seat. My husband’s opinion is that the arm rests that adjoin the middle seat on a three-seat aisle should both be used by the person in the middle seat.

Thoughts? Is there an etiquette answer here?