Are there actually only 17 interior ministry ninjas in the Game?

The second Hirata memory is genuinely awful aside from the Owl boss fight.

where is the key to get in. i did the whole walk through but somehow i didn´t get it

In DragonSpring I can't use my grappling hook, but in Estate Path I can.
That means there is no way off the cliff. I just fall to my death every time. All the Remnants jump off to the limb below that has a place to grapple to but the animation doesn't look like it grappled, but just landed on the branch, which I can't do. The guide above says grapple down.
What am I missing?

Hirata Estate is just Japanese California.

Just started a 2nd playthrough to get the 2nd Hirata Estate memory. Got the eavesdrop opportunity from Emma and the Sculptor right at beginning, where I have yet to defeat the Chained Ogre and go to Hirata for the first time. Will it matter I did this eavesdrop now and not way later after defeating Owl at Ashina Castle?