Forums

OverviewFelgo 2 Support (Qt 5) › How to sync Sqlite database to online server

Tagged: , ,

Viewing 10 posts - 1 through 10 (of 10 total)
  • Author
    Posts
  • #19710

    Felgo User

    Hello,

    Does Felgo have any sort of mechanism that would help with syncing a local sqlite database with a server to allow users to use a database-dependent app on multiple devices? If not, do you recommend any that work well with Felgo / QML? I can use WebStorage for syncing various settings but I don’t see anywhere if that helps with sqlite.

    Thank you,
    Eric

    #19711

    Amir
    #19742

    Alex
    Felgo Team
    #19751

    Felgo User

    Thank you both for your reply but I think I might be confused at a more fundamental level. I come from a background of web development so I approached the app from a similar point of view. Below is a brief description of the app with a request for more information on how you feel I should proceed.

    At its foundation, the app stores data the user inputs. Although most of the data is simply text and integers, some of it will come in the form of saving images. My original thought, which is currently being implemented, was to create a sqlite database as I would have done for a website or app to be only used on a single device (no online syncing required). The database is normalised (currently into about 12 tables) and data is stored there (images stored in BLOB format).

    My problem is in allowing users to sync their data between devices. As I currently understand it, which may be very wrong, if I use a sqlite database, I will basically need to sync the entire database as a file any time a user updates it (I could use Google firebase as my online storage option). The problem with this is that if the database grows, it can start using up a lot of bandwidth as it gets redownloaded over and over. I need the data to be available offline so the full database would have to stored locally and only updated remotely for syncing.

    A different option is to use the WebStorage utility (or Firebase alternative). I originally understood this as simply a utility to store simple key-value pairs. However, I’ve seen examples in Firebase where the Firebase version of WebStorage had more complicated getValue calls with conditional statements. Is it possible to use WebStorage with conditionals? Would it be terribly resource intensive to have possibly thousands of key-value pairs that are searched over where only a few hundred are returned?

    I’m really simply looking for help on how to approach this problem as I don’t think I understand the options too well. I’ve tried reading online help but most of it goes into details for people who already know the fundamentals and have a general idea of what they need to do to accompoish their goals.

    Thank you very much for any help you can provide. If you need more information about what I need, please ask.

    Thank you again,
    Eric

    #19753

    Günther
    Felgo Team

    Hi,

    data that is stored to a custom local database can not quickly be synchronized with a web-server. Of course, you could transmit your whole database file,  but as you also intend to store images this is definitely not a bandwidth-friendly solution 😉

    Synchronization generally also requires to manage “freshness” of the data, which in your case would mean to check which SQLite database file is newer (the local one or the server one). This is only possible for the whole database, unless you additionally use creation timestamps for individual parts of your data model, which you then compare to server values to only sync the parts of the database that changed. So in short: Not an easy topic.

    Maybe there are SQLite syncing frameworks or services available which can cover that. However, I would strongly recommend to use other storage solutions like Firebase or the WebStorage. Both the Firebase Database and Felgo WebStorage support user authentication and data synchronization. They use a non-relational JSON-based system. JSON is easily transmitted via web-services and is directly usable with JavaScript/QML, which is why these systems work well in web and mobile scenarios.

    Felgo WebStorage is only made for key-value pairs, but you can also store complex JSON data structures as a stringified value. Each of the storage entries are synchronized individually whenever a value is overwritten.

    The Firebase Database does not only support storage of complex structures, but also has many other features like queries or cloud functions: https://felgo.com/updates/v-play-2-15-0-qt-5-10-qt-creator-4-5-support-firebase-data-structures-and-queries

    For more advanced queries, indexing and fast searches, Firebase can also be combined with e.g. Algolia Search. We already tried this recently for a client project. We are working on a post at the moment to show how to use Algolia with Firebase, which you can also check out then.

    Best,
    Günther from Felgo

     

    #19782

    Felgo User

    Thank you, Günther, for your reply. Just a couple more follow-up questions, if I may.

    1) Am I seeing it correctly that WebStorage only works with Facebook login? Can user not sync data unless they have a Facebook account?

    2) Again, I may be reading this incorrectly, but does Firebase only create persistant data if the user logs in? How would one go about designing their app if they want the user to have the option of syncing their data or not? Does there need to simply be if statements throughout the code that checks if Firebase is active and runs different data query code if so? Or if Firebase is used, do all users need to be logged in to get any persistent data?

    3) What would be the best approach if I want to allow users to choose whether or not to log in to have synced data? I can certainly convert my sqlite database to use Firebase (or WebStorage) but I’d rather not force any user to log in if they don’t want.

    Thank you again for your time, it’s helping a lot.

    Best,
    Eric

    #19784

    Günther
    Felgo Team

    Hi,

    1) Web Storage + Facebook:
    The WebStorage is a Felgo Game Network service, which relies on the Game Network user authentication. By default, each unique device automatically registers an own account. The easiest way to log into the same account on multiple devices is with Facebook, as the Game Network API and views can handle this use case out-of-the-box. You can also integrate a custom user authentication with the Game Network API to use an account on multiple devices. This is a little more effort though.

    2) Firebase always stores persistent data in the cloud. Also, if FirebaseDatabase::persistenceEnabled is selected the data is also cached offline if there is no connection. You can not deactivate syncing for Firebase. Firebase login is required to maintain and check permission rules to only allow querying data that users are allowed to read. Authentication has nothing to do with database and storage otherwise.

    3) You can set up your login procedure however you like. Felgo Game Network and Firebase are simply tools that you can use for e.g. authentication and cloud storage/database. The game network already auto-creates user accounts for each unique device, but you can also create a similar auth approach with Firebase. However, when using these services all your data will be stored in the cloud primarily.

    If you intend to keep your local database, you will have to introduce your own synchronization – with copying of data from your local database to the cloud.

     

    #19801

    Felgo User

    Thank you again for your replies. I have one more questions. From everything I see on Google’s Firebase site, it’s only available for iOS and Android. If I use v-play to create a desktop version of the app, would Firebase be available?

    Thank you again,
    Eric

    #19804

    Günther
    Felgo Team

    Hi,

    most of the Felgo Plugins mainly support mobile platforms like iOS or Android. They are made to integrate native iOS and Android Frameworks and provide easy-to-use QML APIs.

    When using these APIs on Desktop, you should get a plugin notice if this is the case. Function calls then fail silently so the app can still be UI-tested on Desktop.

    Best,
    Günther

    #19807

    Christian
    Felgo Team

    Hi Eric,

    we can add Desktop support for most Firebase features if you need them as part of the tech support packages we offer. Please contact us here for a quote and to specify your requirements.

    Best, Chris

Viewing 10 posts - 1 through 10 (of 10 total)

RSS feed for this thread

You must be logged in to reply to this topic.

Voted #1 for:

  • Easiest to learn
  • Most time saving
  • Best support

Develop Cross-Platform Apps and Games 50% Faster!

  • Voted the best supported, most time-saving and easiest to learn cross-platform development tool
  • Based on the Qt framework, with native performance and appearance on all platforms including iOS and Android
  • Offers a variety of plugins to monetize, analyze and engage users
FREE!
create apps
create games
cross platform
native performance
3rd party services
game network
multiplayer
level editor
easiest to learn
biggest time saving
best support