Irene
Don't be too disheartened, these things happen with zebra fry, random deaths occur and you'll never know why.
I have had my time with zebra fry random deaths, and find it better to leave the fry in the parents tanks for as long as possible, i only move zebra fry to a grow out tank if the parents tank is looking crowded, at least a couple of months for that, my fry do much better that way.
Do not worry about the fry remaining in males cave, i have had two broods of fry in the males cave, the female visiting to breed and the male hatching out a third brood, i had to remove the fry from the cave eventually when i found a bunch of eggs outside the males cave, the fourth brood, and the fry were fine when i removed them.
Infact, i happier when all the fry go back into to the males cave during the day, i never have any problems when they do that.
Your pair will have bigger broods through time, sounds like they are going through a learning curve just now.