I did this for several months. If you check out the Alpine community you’ll see that many people do this. So, it is not a dumb idea. Alpine is a “generalist” distro and comes packed with all the DEs and WMs you want. They also accept package requests and are usually pretty fast about it.
I would recommend using the Edge branch just to have access to the newest packages, but keep an eye on the issue tracker before hitting update. Also, get on their Matrix and other accounts to follow different discussions.
I ended up using Khal too. Hesitant at first, it wound up being good enough for me, especially when it comes to mass importing events. It syncs with my CalDav and phone when using vdirsyncer. 5 to 10 minutes fiddling with the config and haven’t needed to change anything in years.