Powering BeagleBone with 3.7 V LiPo battery

Yes, you need to pull the PWR_BUT on PMIC low when using the battery. You can do it by pressing the power switch also. In case of USB or AC input this is not needed. Hence the difference.
Refer to TPS65217 battery section for more details.