The pattern that Casey describes is also possible within this model, because the API is layered in the same way, with high-level APIs simply being fast-path compositions of lower-level APIs. The feature flags are just fast-paths for a supported set of feature combinations, which itself is a much larger set than the set of “widget kinds” …
The pattern that Casey describes is also possible within this model, because the API is layered in the same way, with high-level APIs simply being fast-path compositions of lower-level APIs. The feature flags are just fast-paths for a supported set of feature combinations, which itself is a much larger set than the set of “widget kinds” (which would simply poke into that set).
Feature flags aside, builder code can consume input and render graphics arbitrarily. It can take full control over the building codepath in the same way. The feature flags are not the only means by which builder code can provide the effects it needs; they are just options for standardized fast-paths that would be duplicative to rewrite or maintain (e.g. clicking/dragging behavior).
Nevertheless, your mentioning of the position as “library user” is why I do not think UI core code should be packaged within a library for general application development; or, I at least don’t think it’s that simple. Sometimes you do need standardization across multiple applications (e.g. picture yourself as an OS vendor). If libraries were to be provided in this case, I’d say that the ideal decomposition is not obviously “UI library”, and instead it might be that the OS provides simple building blocks that offer controls for its features (accessibility, for example), while allowing composition with user code. (On top of this, you might still have high level APIs that are also just compositions of low level APIs).
This is why I wrote a post on ditching the idea that there is any one layer that defines “what a widget is”—instead, there is just a code code layer that has some building blocks for certain patterns of data transformations. In my experience, there’s rarely a single layer that can totally define a high-level idea; the desired high-level effect must be achieved through composition of multiple layers.
Right, thanks for the insight. Thinking about this problem, I started learning about data-oriented designs and I came up with this idea: instead of storing the flags inside the widget struct, we could store them out of band, in collections (arrays or hash maps) of widget references (pointers or indices) for each flag, consumed by functions which operate only on a single flag or something like that, opening up the possibility of adding new features in the future. What do you think about this approach? Maybe it could bring some issues in the layouting phase.
I think what you’re getting at is just mirroring entity component systems (for the record, I don’t actually consider using SoA-style storage for an ECS to be data-oriented in itself—data-oriented thinking has much more to do with first principles rather than a specific architectural choice).
This would work in theory but is unlikely to provide the wins you might expect it to, and it will probably result in a loss of flexibility—namely because you’re adding structure to “features” where structure may not exist: it isn’t obvious that a single feature breaks down into a single batch-processing codepath, especially because many operations in UI are order-dependent and thus serially-dependent.
It’s also unclear if storing the flags out-of-band will be of much use. It is true that it would allow you to look at flags without pulling other per-widget data into cache, but the general structure of UIs in my experience is two codepaths (building + rendering) that often touch all per-widget data.
But, in any case, it actually is unlikely to matter at all, due to the small number of widgets you need active at any one point in time (even with e.g. infinite lists), where all of your widget data is likely able to fit inside L2, and where there are just barely any widgets to begin with. It’s just not a data processing problem worth thinking much about, because there isn’t much data.
If you want the ability of builder code to extend per-widget data (e.g. with its own feature flags, or other stuff), then you can just equip each node with a slot for extra user data, and let the user (builder) code decide how to fill it out and later use it. There’s not much reason to shoehorn user-attached data into the first-class slot of features that the core supports and is aware of.
Thanks again. You're right, I was mirroring an ECS. Unfortunately I was thinking ahead of time if an ECS would fit in this context, but first I actually need to start implementing your ideas.
Can't wait for what's coming next. What are you planning to do?
Yeah for sure. You can only organize data for bulk efficient data transforms once you know what the set of transforms you need is, and what the shape of the problem looks like. That may vary dramatically depending on the problem, so there’s no “bag of tricks” here, you just need to explore the shape of the problem and then do another pass of data organization. That’s why I choose to keep my data organization and types very simple when in an exploratory phase—you can use simple bucketing mechanisms to figure out the shape, and then use that to inform you of what the really tight version would look like.
I still have a lot to write about—I want to get to rendering, infinite lists, text input, panel trees, and so on. There’s a lot to sort through and I haven’t organized it all yet, so not too sure what’s coming next, but it’s probably stuff like that!
The pattern that Casey describes is also possible within this model, because the API is layered in the same way, with high-level APIs simply being fast-path compositions of lower-level APIs. The feature flags are just fast-paths for a supported set of feature combinations, which itself is a much larger set than the set of “widget kinds” (which would simply poke into that set).
Feature flags aside, builder code can consume input and render graphics arbitrarily. It can take full control over the building codepath in the same way. The feature flags are not the only means by which builder code can provide the effects it needs; they are just options for standardized fast-paths that would be duplicative to rewrite or maintain (e.g. clicking/dragging behavior).
Nevertheless, your mentioning of the position as “library user” is why I do not think UI core code should be packaged within a library for general application development; or, I at least don’t think it’s that simple. Sometimes you do need standardization across multiple applications (e.g. picture yourself as an OS vendor). If libraries were to be provided in this case, I’d say that the ideal decomposition is not obviously “UI library”, and instead it might be that the OS provides simple building blocks that offer controls for its features (accessibility, for example), while allowing composition with user code. (On top of this, you might still have high level APIs that are also just compositions of low level APIs).
This is why I wrote a post on ditching the idea that there is any one layer that defines “what a widget is”—instead, there is just a code code layer that has some building blocks for certain patterns of data transformations. In my experience, there’s rarely a single layer that can totally define a high-level idea; the desired high-level effect must be achieved through composition of multiple layers.
Right, thanks for the insight. Thinking about this problem, I started learning about data-oriented designs and I came up with this idea: instead of storing the flags inside the widget struct, we could store them out of band, in collections (arrays or hash maps) of widget references (pointers or indices) for each flag, consumed by functions which operate only on a single flag or something like that, opening up the possibility of adding new features in the future. What do you think about this approach? Maybe it could bring some issues in the layouting phase.
I think what you’re getting at is just mirroring entity component systems (for the record, I don’t actually consider using SoA-style storage for an ECS to be data-oriented in itself—data-oriented thinking has much more to do with first principles rather than a specific architectural choice).
This would work in theory but is unlikely to provide the wins you might expect it to, and it will probably result in a loss of flexibility—namely because you’re adding structure to “features” where structure may not exist: it isn’t obvious that a single feature breaks down into a single batch-processing codepath, especially because many operations in UI are order-dependent and thus serially-dependent.
It’s also unclear if storing the flags out-of-band will be of much use. It is true that it would allow you to look at flags without pulling other per-widget data into cache, but the general structure of UIs in my experience is two codepaths (building + rendering) that often touch all per-widget data.
But, in any case, it actually is unlikely to matter at all, due to the small number of widgets you need active at any one point in time (even with e.g. infinite lists), where all of your widget data is likely able to fit inside L2, and where there are just barely any widgets to begin with. It’s just not a data processing problem worth thinking much about, because there isn’t much data.
If you want the ability of builder code to extend per-widget data (e.g. with its own feature flags, or other stuff), then you can just equip each node with a slot for extra user data, and let the user (builder) code decide how to fill it out and later use it. There’s not much reason to shoehorn user-attached data into the first-class slot of features that the core supports and is aware of.
Thanks again. You're right, I was mirroring an ECS. Unfortunately I was thinking ahead of time if an ECS would fit in this context, but first I actually need to start implementing your ideas.
Can't wait for what's coming next. What are you planning to do?
Yeah for sure. You can only organize data for bulk efficient data transforms once you know what the set of transforms you need is, and what the shape of the problem looks like. That may vary dramatically depending on the problem, so there’s no “bag of tricks” here, you just need to explore the shape of the problem and then do another pass of data organization. That’s why I choose to keep my data organization and types very simple when in an exploratory phase—you can use simple bucketing mechanisms to figure out the shape, and then use that to inform you of what the really tight version would look like.
I still have a lot to write about—I want to get to rendering, infinite lists, text input, panel trees, and so on. There’s a lot to sort through and I haven’t organized it all yet, so not too sure what’s coming next, but it’s probably stuff like that!
You are absolutely right. You give very simple and powerful insights, which are the hardest to find. Thank you so much.
Text is one of the most interesting topics to me, I'm looking forward to the next posts. You are doing a very good job, keep it up!