Last week I've released gogo. It only has the form handling component and does not show how to set up the app completely, but the form handling alone should showcase the efficiency of the approach. Server-side validation is not without it's downsides, however for the prototyping phase nothing could beat speed and consistency and both are provided.
Not all of the dabdab forms are handled this way, however the reason is purely historical, all new features are developed this way
Unexpected footgun with go templates: it's possible to define two blocks with the same name (which is fine), however one would silently override the other one even if they're both located in template that do not depend on each other in any way.
Dabdab has lost an ability to add notes and todo items to the messages because of that, but now everything is back
API implementation for dabdab was quite simple. I wasn't ready to implement a full-featured api with all the niceties, because there many security and convenience related questions, however a per stream api access was a rather trivial exercise.
Some of the reasons apis are hard to work with are
I've solved the first point by not requiring api keys in the first place - a share link with api access works like a slack hook that allows to post messages with a simple post and it was ok since the url was unique.
Second point is solved by generating a real curl call that can be used to post a message. And not only that, all the values in the payload are also generated to be random.
The remaining point was about testing. The usecase for such an api is some sort of automation that gathers data from time to time and user wants to store the data somewhere and render some nice graphs. I didn't want to run a cron and used sister service webhks.com to trigger a webhook every now and that and the webhook post weather updates
More serious progress on dabdab:
I want more, to be precise, I want
I've found another reason for a slowdown on the home page - stimulus actually launches the component to render the timeline even though the element is hidden. It kind of makes sense, but I just don't see a clean way to avoid it
One more evening spent on optimizing the performance of dabdab.org.
This time I've extracted all js related to visualisation into a separate js bundle + added a prefetch link tag to make sure this file is cached by browser. The size is still pathetic though, however this allowed me to return echarts since go package produces a really bad quality results.
As a next step I'm thinking about ditching uikit js components and rewrite few that matter to avoid bringing in 200 kb of useless code to the user
I was not totally happy with the frontend performance of dabdab and decided to spend an evening looking into it. Some of the changes:
The timeline package that I use is a real waste as well, it takes a whole bunch of dependencies with it that are worth of hunderds of kilobytes. The other offender is uikit framework, since many of the components that should be purely css like a grid for example, require js for no apparent reason.
From js side one potential fix is to use bundle chunking and load all the js only on the pages that need them. For example, symfony supports it, rails supports it too
With the change password form on settings I decided to formalize the pattern I use for the forms everywhere on the site. No surprise, the concept is the same as with django forms, less abstraction though - I'm not doing field types, automatic form generation and such. Every abstractions adds complexity, and at the moment it's not obvious whether it's worth it
The latest share stream feature required quite a few changes on the app to enable all the flexibility. Under the hood the same template is used to display a stream (or a message) no matter how it's being accessed. That's quite a lot of conditionals, but usually the problem is not with having conditional blocks but around having cryptic conditions that power them. I've seen it some many times - you open a template and see something like {{ if and .OneCondition (and .User .SecondCondition) (not .AnotherRandomCondition }}. The only feeling you get is the one of desperation since there is no reasonable way to change this condition without breaking things, especially if the template is used in several places.
The approach I took was to switch from ad hoc conditions to a capabilities based approach. For a given stream there is a set of flags which defines what a user can do with a stream - can they read? Can they add changes? Is it their own stream? With this approach the complexity in the templates goes back to a bearable level.
Another point to address is to synchronize the conditions in the template with the ones in the backend code. What I did in my case was I extracted all the code that calculates user capabilities into a single function that does all the calculations and use it both to control the behavior in the templates and the behavior on the backend. With that move the code became even simpler
dabdab.org supports markdown with the help of goldmark. It's really hard to overestimate the usefulness of a decent parser. Since introducing image uploads I was really worried about hardcoding uploaded links into the message text. I assume message text almost immutable - it's weakly structured and any text updates will be quite painful.
What that means in practice - message text should contain as little representation details as possible and image urls is just one of such details. From now own any media upload will result in a simple <uuid>.png url, which will be properly resolved at the rendering time. The parser was really useful there since it's trivial to walk the parsed tree and update links if necessary
As it turns out, a textarea element treats dropped files as strings instead of files and correct way to fix the situation is to simply use it's parent element
What makes SPAs compelling is their tolerance to the latency. It's always possible to add a spinner, cache the state and cover backend deficiencies this way. It sounds just right, but at the same time what you really want to provide is a really slick experience to the user. When stated this way, any big latency should instead be exposed to force developers to act on it.
However, when the latency is always under control, it's very possible to use old school methods with page refresh which is less complex and allows to use the full power of libraries and resources on the backend
Officially wasted an evening chasing a red herring of a simple markdown editor. There are very few projects out there and all of the drag a full blown markdown parser, code mirror, ton of styles and optionally the whole web font in order to draw an enhance textarea, which sounds totally not compatible with the idea of keeping dabdab as fast and as light as possible.
The ideal component for dabdab would be very similar to what github minus preview tab, since the whole idea of a toolbar would be to have the buttons that would remove the necessity to switch the keyboard on the phone in order to find symbols to render a quote or a bold font.
The other awesome feature github has is their image upload functionality, I want that as well
The concept of middlewares in gin is really powerful. Of course it's nothing exceptional, most of frameworks have it, but still. For example, custom error reporting is trivially implemented this way.
it's something like
r.Use(gin.CustomRecovery(func(c *gin.Context, err any) {
userData := auth.GetUserData(c)
admin.NotifyPageFailure(c, err, user)
}))
The only limitation is that a beautiful error message that gin prints is compiles by private methods and is not passed to the callback. If you want the same data you'll probably do some copy/paste
One of the interesting problems of every public service is that the amount of possible states a given user can land in grows exponentially over time. In many cases the easiest way to reproduce the situation is log in as this user. However it goes against privacy considerations of course. On dabdab private notes should be treated as private by developers as well.
An approach one could take is to allow to impersonate a user and fake all the content of the user while keeping the structure. If a user has a stream with three records, one of which has a youtube embed and the other has a list in it, generating some stream with the fake content which would have one message with embed and another one with a list should be enough for troubleshooting while keeping user data private.
Usage report so far: works really well with reading and travel notes, same efficiency as writing on the margins, however no need to digitize them afterwards and there is no difference between paper and electronic books
What’s really missing is some sort of offline support to make sure one could take notes on a plane for example
Google authentenicator support seems to be a solved problem, the package is there. I think the whole feature can be added in a couple of hours
The 0.0.0.0.1 version of landing page was deployed. It's the opposite of perfect, but that's fine, the very first step is always the hardest because that's an actual innovation, all subsequent steps are just optimizations even if the change the page completely.
I still haven't decided on ways to communicate between different users. Private streams do not need communication, but the fact that a stream can be public implies some form of discussion. I don't want to implement usual comments flow (be it with threads or not). I guess a response style like from twitter could work? In this case the author is still notified and at the same time all the comments still physically reside on the commenter's blog.
One way to go about it would be to allow to create derived messages. It could work as a comment and at the same time enable a number of use cases. E.g. you have a stream with books to read and create a derived messages with a stream with reviews for any book you've read.
Just realized that mailjet is replacing links with it's own. That's fine per se (open/click tracking etc) but the domain they have (0gxsg.mjt.lu in my case) looks like 100% phishing.
Javascript packaging solution is really broken. Despite having multiple package managers it's still not trivial to import a package times. I really want to render a timeline for my streams and decided to use vis.js for that since it's the most popular open source solution. Not only importing vis-timeline was not enough, I had to hunt for a solution. It's basically this and don't forget the styles
Compare this experience with goland where I'm yet to encounter a package I don't know how to import.
The library itself looks cool though!
Goldmark markdown parser is surprisingly good and ast it builds enables all sorts of nice features that would be hard to implement otherwise. As an example, youtube embeds feature wasn't that hard precisely because I didn't have to parse markdown or do any hacks with regular expressions that won't be any good any way in addition to be very fragile.
Ast allows me to
Of course there is a bit about parsing links themselves and adding proper embeds, but it's really easy compared to markdown parsing
Sometimes it's useful to think about the limitation of the data model. Every developer can remember both cases when the model was too limited and when it was too over-engineered, the former case is hard to fix because the data is already used in many places, the latter because, well, the data is already used in many places.
One limitation of current dabdab data model is that the handle (can3p in my case) is tightly coupled to the user account. It may seem natural, but doing it this way automatically makes impossible to have shared handles (just imagine having a corporate space on dabdab) or makes it very complicated to transfer the handle between the accounts. Also, it implies that one user account can have only one handle, which may be true for some time, but it's super hard to fix at the later stages.
I've been designing dabdab in a way to solve/avoid many limitations of existing systems and that's one more worth solving
Adding support for charts was quite straightforward. Googling "charts golang" gave me a link to the go-echarts repo. Docs are nonexistent but it works and what's even more surprising eventhough it's used against echarts version 4.8.0 which was released in 2020, it works just fine with the latest version of the library.
Why use go-echarts? I try to postpone doing an SPA as long as I can get away with it and the package provides a nice way to setup options for the chart rendering. In order to render the chart I use stimulus which provides a nice way to bootstrap components automatically.
All in all, most of the time was spent on fighting with echarts configurations than on anything else
Modern web development feels like cheating sometimes. Just last week I was told about gap css property and today when I wanted to keep the aspect ratio for the block I've stumbled upon aspect-ratio property that does that trick without any hacks that were needed before
Cmd-Enter doesn't work on Mac OS X because I use keyup event and apparentely this specific event is not fired when the meta/cmd key is pressed on mac os x
Example issue elsewhere: https://github.com/electron/electron/issues/5188
Suggested fix was to start listening to keydown instead
on keydown[(ctrlKey or metaKey) and key == 'Enter']
call my checkValidity()
if the result is true
submit() me
otherwise
call my reportValidity()
Small project is done in small steps. However it's nice to see how new primitives fit in together. This time I've added email notifications for myself to get alerted whenever a new user signs up / new stream gets created. I'm sure this can work for a long time before I'll have to disable new stream notifications first and new user notifications after that.
Fun fact: even with zero users, I already see bots probing the service in hope to find wp-admin.php
Also, user page was broken, due to template migration to new types, this is fixed now
htmx is really bare bones and prefers replacing html as opposed to updating fields with javascript. When done this way a structure similar do django forms inevitably arises if one tries to think about code reuse
One cool idea from today - temporarily share a stream like it's done with location on Telegram. You can go even further and make streams "buyable" with an option to buy access for a few hours
Stream Product updates has begun
Usual approach for note taking apps is to have a collection of pages / blocks that can be linked together. Although it's possible to get really far with this approach, there are several fundamental problems there:
All three points really bothered me every time I tried to use a note taking app. The maintenance burden is especially daunting. While it always starts very simple, bits of information add up very quickly and quite soon most of the time with notes is spent on organizing them instead of writing them.
In addition I argue that in majority of cases we don't need any structure in the first place. Take gmail or slack as an example. The fact that information is scattered across unrelated emails/threads/channels absolutely does not make it hard to use them to keep and extract the information. It's seen as an antipattern by many people but what if that's a feature?
Another aspect - in many cases notes we are taking do not need to be read again and analyzed. In some of the cases we just need a place to write things down because the sole fact of writing things down forces us to structure the thinking and that's what is valuable, not the notes themselves. In other cases what we want to record is just a fact - a change in the temperature, distance that we ran for today and things alike.
And let's dive into the last case - recording facts of life. How many times you got sick this year? For how long? If you take such questions what's needed is not a note, but time between events. You could record the dates in a separate note, but that won't help you to compare the ranges or get any other types of insights.
And that's what dabdab is about!
The central concept there is stream. Stream is a series of messages. A stream can have substreams and can be public or private. You can have as many streams as you like. That's all to it!
One example of stream could be this blog which is a series of streams.
Or you can create a stream "Trips" where you would drop any information about your holiday plans and then you can create a substream for a specific trip you make to collect all the related information in one place. Given such structure it's trivial for dabdab to generate a graphical timeline which would be useful to extract insights from it.
Messages are not only a block of text. For every stream it's possible to define an arbitrary number of custom fields that will be able to have different types and allow to track interesting bits of information for your stream. If you get sick, you can track your temperature, if you're traveling you can record your location in order to reconstruct the route later. Messages are really cheap, no need to think them through if you don't want it!
There are two more concepts directly related to messages - tasks and facts. As was mentioned before, in many cases the message itself is not as useful as the decisions made based on it or their other properties (timestamp, fields). Let's say you take a note of you and your partner discussing future vacation plans. Fact is just a short note that's attached to the message but can also be later observed alongside the stream. Based on the message you can for example add a fact "She doesn't like posh hotels" and have it visible in a separate list attached to your Trips stream. If you decided that it's time to book some tickets that evening you can create a task to book them. The task will be separately visible for your stream too and you can easily work through the task list to get things done.
Streams are a very powerful concept. Developers can imagine writing a bot that would maintain a stream related to uptime of their service or acting as a collector for the feedback they get via some channel. Scientists can keep research notes and create substreams for experiments, the possibilities are infinite.
Since this blog is pubilc you may wonder about comments, likes and all other usual aspects of social platforms. My current idea is that you probably don't need it there. Twitter is uniquely position to propagate the information at maximum speed, dabdab is made to organize thinking process. It doesn't mean that social interactions won't ever be a part of the platform, but we will for sure try to implement them in a different less harmful and addictive.
Stay tuned!
Dabdab is a new name for the service.
What does it mean? In short - not much, the name is random, meaningless and does not explain the nature of the approach / concept in anyway. The only requirement was to make sure the name is easy to pronounce.