Hi Francois,
On iOS this is supported out-of-the-box, on Android this is only possible with certain launcher apps installed on the device, which is currently available for Samsung, HTC, some Sony, Apex and Solid launchers.
How are you sending your pushes out? If you’re using the OneSignal REST API, you can just add the ios_badgeCount key to your payload, as described here in the documentation.
Hope this answers your question!
Best,
Alex