It is of course French, and I am not an expert in that language; but it is what the croupier calls during games of roulette, if I remember rightly.
In the poem, it means they realized that everything was now finished; they would never be able to fulfil their dreams.

Reading the verse, this expression seems to be used in the sense "the game's up," "there are no more chances or reprieves." When I suggested "there is no going back" above, it was in the sense that when fate intervenes a person cannot go back in time and watch his/her children grow up, accomplish things left undone, or experience what her or she has not yet experienced. It's about fate, rupture (and one supposes death).

die is cast
Prov. A process is past the point of no return. (The die is one of a pair of dice. The cast means thrown. This phrase [in Latin] was said by Julius Caesar when he crossed the Rubicon with his legions, starting a civil war.) After that speech favoring reform of the education system, the die is cast. This is now a campaign issue. The die is cast. There is no turning back on this point.
books.google.es/books?isbn=0203015347 -

Antony Kamm - 2006 - ‎History
... form of a metaphor from the game of dice: 'The die is cast,' reflecting the modern croupier's cry, 'Rien ne va plus,' or else 'Let the die be cast.' Whichever it was ...
clockrootbooks.com/.../vqr-on-rien-ne-va-plus-our-l...‎

01/04/2010 - Rien ne va plus, the phrase that is delivered in roulette when 'the game becomes fate,' is a central metaphor for Karapanou because her novel ...

par ex.