This was however not easy to setup at all because geofencing is not really designed with that use case in mind in APM::Plane. As discussed in this bug: https://github.com/diydrones/ardupilot/issues/2984 , in arducopter you can simply set an altitude limit but not in Arduplane.
So, what you need to do is the following:
You're not done though, now you have to upload a useless geofence area or nothing will work. As per http://plane.ardupilot.com/wiki/geofencing/ , I uploaded a geofence as big as the US, and put the geofence return point in the middle of the ocean.
The end result is that the geofence return point is too far, so when guided mode kicks in, it looks for a rally point, I have none, and then falls back to the RTL/takeoff point, which is what I wanted.
Sadly, though, I was only able to setup that useless geofence area with mission planner.
So there you go, this can be made to work, but apparently you have to use mission planner (which means Virtualbox for me since I'm on linux), and you need to setup a huge polygon, a gence return point that is very far away from where you'll ever fly, and make sure that you do set FENCE_MAXALT or FENCE_MINALT won't work (which seems to be a bug, as of AP 3.4).