Did you never know, long ago, how much you loved me—
That your love would never lessen and never go?
You were young then, proud and fresh-hearted,
You were too young to know.
Fate is a wind, and red leaves fly before it
Far apart, far away in the gusty time of year—
Seldom we meet now, but when I hear you speaking,
I know your secret, my dear, my dear.
The poem is clearly addressed to a young man–or perhaps one is is older now–who was in love with the speaker but never acknowledged his love. She understood his feeling, even without any discussion of it. Now when they meet, she remembers how much he loved her, how she could see and feel his affection.
Basically, this poem is about her beloved showing her so much compassion and love. Also, this is about their relationship after death.