Vidu - Minimal (jamstack) web analytics

About

This project was started for trying/learning the following tools and still end up with something useful (at least to me):

• Svelte javascript-framework (Vidu SPA backend)
• Netlify static hosting
• Netlify Identity for user authentication with GoTrueJS
• Netlify Functions AWS lambda functions for API calls
• Fauna as transactional database
• GitHub Actions to execute build hook of monitored web page to refresh encryption salt every day at midnight
• Netlify Build Plugins to retain encryption salt on normal builds and refresh it if build web hook is executed by GitHub action
• Windi CSS Next generation Tailwind CSS compiler

Functionality

Vidu [see - /Esperanto/] consists of

• A Netlify Function that is included as "tracker" in your web pages to collect (anonymized) user data and sending these to a Fauna database.
• A Svelte SPA which displays all the data in a simple yet beautiful dashboard.

Disclaimer

This project was created for fun and educational purposes.

Vidu, in general, has no limitation on processing page hits. But keep in mind that by using it for highly frequented web pages you will most likely exceed the free plans of Netlify and Fauna.

Fork it. Extend it. It's "unlicensed".

Recommendation

If the above is the case or you've stumbled up-on this repository in search for a alternative web analytics tool, I may suggest you also take a look at these (more feature-rich) solutions:

• Netlify Analytics Netlify's own analytics tool (SaaS)
• Ackee Node.js based analytics tool for those who care about privacy (self-hosted)
• Fathom privacy-first alternative to Google Analytics (SaaS)

Setup

I assume you already have a Netlify account.

Setup Fauna DB and Vidu

1. Create database

If you don't have a Fauna account yet, sign up here. Create a new database by clicking the New Database button and filling in a name in the form that follows:

2. Create a database key

The FAUNA_SECRET for your database can be created in the Fauna dashboard by going to the SECURITY menu and clicking NEW KEY. Make sure to select the Admin role since our key requires full access to the database.

Fill in a name, press Save, and you will receive a new key. Make sure to copy it since you will only receive this key once. Since this key will have full access to your database, make sure to keep it somewhere safe.

3. Deploy repository to Netlify

With your server key ready, you can easily clone this repo and deploy this app in a few seconds by clicking the deploy button. Fill out the form and enter your FAUNA_SECRET. The DB structure will be setup as pre-build task. See bootstrap-db.js for details.

4. Enable Identity

Enable Netlify Identity feature on your newly deployed Netlify site. Otherwise adding users and logins wont work.

5. Configure emails sent during user registration and other actions

Under Site Settings go to Identity - Emails and configure the templates as follows:

Invitation template Path: /email_templates/invitation.html
Recovery template Path: /email_templates/recover.html

6. Set registration to invite only

Under Site Settings go to Identity - Overview set Registration preferences to Invite only. You can then go to the Identity tab and invite/add your first user which should receive a confirmation email.

Setup Tracking

For how to implement the tracking in your web page take a look at the example. Further details below.

Do not forget to add your FAUNA_SECRET to the environment variables of this Netlify web page, too!

Tracking Code

The tracking code calls a Netlify function which pushes the tracking data to your fauna database and returns a bas64 encoded transparent image.

<img src="/.netlify/functions/counter"
      alt=""
      width="0"
      height="0"
      hidden
      decoding="async"
      loading="eager" />

Netlify Function Node Dependencies

See example: package.json.

Gathered Data

• URL of visited page
• Browser
• Browser version
• OS
• OS version number
• OS version name
• Client language
• Timestamp
• id

The id is an anonymized user string composed as follows:

Anonymized ID

To compose the id the last byte of the tracked IP address is removed (the last 64bits if IPv6) and hashed (sha256) together with the above listed and salted data. As the URL is also included in the hash, you wont be able to track the browsing behavior of individuals across your page. See example: counter.js for details.

The 16 bytes random salt is created once during page build by making use of a Netlify plugin(Example: netlify-plugin-handle-mysalt). If a salt was already present it will be restored from cache during a new page build.

Nevertheless, the salt is always kept separated and never stored in the database.

Further Improve Anonymization

To further improve the anonymization of the gathered data you can setup a github workflow(Example: workflow) running a build hook to rebuild your web page every day at midnight (UTC). The above mentioned Netlify plugin will recognize that the page build was triggered by your build hook and in this case a new salt will be created during page build.

• Add a build hook under Site Settings - Build & deploy - Build hooks.
• Name it refreshsalt
• Edit the Example: workflow by adding the created url to it:

run: curl -X POST -d 'refreshsalt' <YOUR_BUILD_HOOK_URL>&trigger_title=Refreshing+salt+by+github+webhook

And that's just about it!