1) The easiest way to do so is buying a new kit or at least matched parts.
Depends. If your controller is programmable and the FETs can handle higher current. If so, you could just reprogram it for more power. If it's not programmable, but have good FETs, you could gamble and do a shunt mod.
Safest option however is a controller designed for higher power.
2) Upgrading the BMS from 50A to 200A will potentially decrease my Ah (autonomy) because I will be able to output more current to the controller, should it be rated high enough to feed a 8000w motor (generally 88-200A, taking peak amps into account).
Upgrading the BMS does nothing in terms of battery capacity. The BMS current limit does not actually limit the current the controller can pull, it simply disconnects the battery from the controller if you overload it for an extended time. When pulling more current from the battery, the usable battery capacity does generally decrease, but this is not because of the BMS current rating.
3) If my controller is rated at 50A, nothing will change, it will prevent any more current anyway. My 88A motor won't make its advertised power in all conditions (hills..) because I can only draw 50A anyway. It should approach its optimal perf on flat roads though.
If your controller have a max battery draw current of 50A, changing the BMS does absolutely nothing. Do however note that battery and phase currents are two different things, and your motor might still receive 88A (or a lot more) even though the controller only pulls 50A from the battery.
4) My driving style and hills may incur a dangerous heat increase for the battery.
This depends on a lot of things. Battery cells, how the battery is designed and located, the outside temperature, how long the hills are, how steep they are, how heavy you are. In general with quality cells that are not being overloaded you shouldn't have any safety issues, but it's still recommended to have a BMS that disconnect if the battery gets too hot, especially if you intend to load the batteries higher then the pack was designed for.
5) My battery may blow if the cells used are not up to the task (lack of cell quality)
Not likely. They will sag like crazy and heat up a lot, but as long as you keep an eye on the temp you're usually not risking an explosion. If possible i'd try to find out what cells is used and stay below the max current draw. If it's a cheap Chinese no-name pack, expect it to already be overloaded at "rated" max current draw. Use a BMS with temp sensor for safety.