Transcript

00:00:00 - Speaker 1: Infinite canvases are essentially like a different document format. The screen represents a camera that’s floating above a surface, and there are things on that surface, and those things can be anything. You can move them, you can duplicate them or resize them, and that each one of these types of thing on the canvas also has its own rules about how it can be changed.

00:00:26 - Speaker 2: Hello and welcome to Meta Muse. Us as a tool for deep work on iPad and Mac, but this podcast isn’t about muse product, it’s about the small team and the big ideas behind it. I’m Adam Wiggins here with my colleague Mark McCrannigan. Hey, Adam. And our guest today is Steve Ruiz of TL Draw. Hello. And Steve, I understand you’re a little bit ahead of me on the fatherhood journey. You’ve got a 315 year old. What do I have to look forward to in the coming couple of years?

00:00:55 - Speaker 1: Oh man, a kid is what, like 1.5?

00:00:58 - Speaker 2: That’s right.

00:00:59 - Speaker 1: Eventually they will start drawing, they’ll start playing with words, and they will grow ever more interested in iPads, and also, everything will be an iPad, every computer, every device, everything with the screen will be an iPad. That’s my prediction, yep.

00:01:17 - Speaker 2: Yeah, that makes sense. Yeah, already is the case that trying to keep screen time limited is a challenge. Now, of course, we can also look forward to having built in beta testers for our software. You got a chance to do that yet?

00:01:29 - Speaker 1: Yeah, actually, my daughter has probably spent more time with tealra than anyone else, and I’ve learned quite a lot by watching her kind of poke around and try and draw, try and make different things. She really likes arrows, but the touch targets are too small and mobile. That’s one thing that I need to work on.

00:01:46 - Speaker 2: I’m reminded of a scene in one of these Steve Jobs biography movies where I believe it’s his daughter comes in and tries out Mac Paint, and they sort of show this maybe hypothetical idea that he was sort of inspired to make software and a computer generally that was easy enough that a young person could be creative with it.

00:02:04 - Speaker 1: Yeah, I mean, if you have, uh, they’re kind of hard to schedule or bring into the office. I don’t think the agencies do 3 year olds, but toddlers are excellent beta testers. They won’t tell you what’s wrong, but you’ll definitely see it.

00:02:19 - Speaker 2: So you’re working on TLDraw. Tell us about that project and your background that brought you there.

00:02:24 - Speaker 1: Yeah, so Tal Draw it started out as like an open source project, I guess it still is an open source project. It grew out of kind of earlier open source work that I did with a library called Perfect Freehand, which is an algorithm for creating like digital ink, so pressure sensitive or variable width lines, takes a whole bunch of input points, and comes up with a whole bunch of output points that kind of describe a shape that surrounds the input points. So, perfect freehand, I kind of started working on it pretty publicly on Twitter. So posting all these in progress GIFs and talking about the different ways that I was trying to solve this problem of doing pressure sensitive slash variable width lines.

You can’t really do that in the browser, like it doesn’t have any primitives for like lines that change their width. So, hence the whole like journey into trying to figure out how to do that kind of programmatically myself.

It works, it works super good. I would recommend it if you have anything to do with the browser that needs to have a kind of a cool ink. And it’s being used all over the place. It’s used in Draw.io, it’s used in Excali Draw, it’s used on, I think NextGS Live uses it in their product.

It was one of these situations where The status quo for like a pencil tool or a pen tool was like so poor on the internet or on the browser especially. That any improvement sort of would have been well timed and perfect freehand just happens to be an improvement that is pretty good. You should use it. That’s what I say.

So, I’d been working on perfect freehand, I’d been doing some integrations with like other diagramming tools like cala Draw and I guess in along the way of making perfect freehand, I built a couple of these. Canvas playgrounds almost, places where I could test this thing out or try out the different algorithms, see where it was going right or wrong. And I’ve done that a couple of times for Perfect Freehand, also for like a kind of an offshoot of this project called Globs. But anyway, I just made enough of these sort of infinite canvas editors that once Perfect Freehand was pretty much done, I started working on that kind of a framework.

Originally it was gonna be like a programmable design tool that I could use both together with direct manipulation, but also with like, you know, programming, in order to help me figure out problems like perfect forehand or problems like cool arrows or something. And once I started posting that, On Twitter, saying like, oh cool, I’m looking, I’m kind of making almost like a figma that you can program, or a figma that you can put anything on the canvas that you want, and I’m doing it in React. That story suddenly got pretty popular. Like I had folks reaching out to me and saying like, hey, we’re building something that kind of needs this kind of canvas. Can you open source this or can you share this or can I hire you? So based on that attention, I was like, all right, well, maybe this is worth proceeding with. I took some time off between jobs. I was gonna start a job actually at Adobe, and I thought, OK, I’ll take some time off in between and I’m just gonna work on this project, make it open source, spend 6 months on it, and then, yeah, see what happens. Worst case scenario is that I’ve made something really cool open source, worked on something interesting in this space, and I should say there’s really very, very few open source kind of canvas UI type of tools out there. And none of them that were using this kind of react driven canvas.

00:06:04 - Speaker 2: It’s interesting that you got that initial demand, you might say a product manager lingo might be it’s sort of market validation, you know, don’t build it until you already know people want it. Even before you got into the project itself, did you have a sense of why people wanted it or had they tried to build it themselves and discovered it was hard? Had they not built it and they just hoped someone else would do it for them? It just they liked your other work and they just thought if you did a good job on this other library, maybe you do a good job on this library too. What was the core of their demand, I guess.

00:06:36 - Speaker 1: Yeah, my theory about this or my guess about this is that this was 2021, so not too long ago, 18 months ago maybe. We were still middle of the pandemic, seemed like everyone had started using these apps like Miro or Mural or I think Fig Jam had just been released and The place for online collaboration was starting to just be the sort of the whiteboard, you know, abstractly like this 2D kind of surface that you could move around on and interact with people and co-create this type of surface.

And the tools in that space were pretty mature, like again mural fig jam was new, but it was already, you know, built on good bones of FIMA itself. And so, As normally happens with like successful general products, like you start seeing different teams wanna carve off verticals for that, say, like, OK, cool, Mirro is great. Mirro is for everyone, Miro is for everything. What if it was just Mirro for project managers? What would that look like? Can we steal a billion dollars off of Canva? Can we, you know, take a little bit of that giant market from FIMA or something? The issue there is that all of these tools, pretty much anything. That wants to use that kind of canvas UI, it’s a tough engineering problem. And there’s like a ton of uh functionality maybe of like a canvas like that that is like, it’s almost like a text editor that it just has to be there in order for it to feel complete. And if it’s not complete, it’ll feel broken. But if it is complete, if it has those like table sticks features and all that, then congratulations, no one will notice because they expected them to be there from the beginning. And maybe you ran into some of this with Muse as well like. Selection should just work, like Undo reader should just work, the cloud stuff should just work. And some like really gnarly, like logic puzzles involved in these type of things that we take for granted in something like Mirro or fig jam. Yeah, like they’re they’re just gnarly problems. Never mind the whole like, how do I render this thing. So when I started saying like, look, this is something that you can use as like a starting point, where all those problems are gonna be solved. Now it’s just about picking like, what do you want to put on the canvas, how do you want these things to interact with each other. I think that story of giving someone like good open source starting place, that was like the compelling story. It’s just like the same with maybe like prose mirror or a code mirror. Text editors are super hard, no one wants to build one themselves. I mean, I don’t want to build a text editor myself ever. But I do want to make apps that include them, and I do want to do stuff with them, and if I can do that on open source work, then all the better. So, I think that’s where the validation started clicking in, or like pre-validation. I didn’t have an idea really, but I did notice that there was a demand for this kind of thing, because, yeah, it was becoming more of like what software looks like, especially around collaboration, like it looks like a canvas.

00:09:30 - Speaker 2: Fun little parallel story there from the Hiroki days, which is a very early version of Hiroki that was mid-2000s, 2007, 2008, had a text editor in it, but yeah, we had to implement that from scratch in the browser and you know, pretty quickly you exactly as you said, all these table stakes things you just expect to work, particularly in programming editors, you know, we had to implement it and that’s not the differentiated or wasn’t the differentiated part of the product at the time.

But it wasn’t long after that that the AC editor, I believe it was, was kind of the first really solid open source in the browser code editor, and that seemed to unlock a kind of explosion of people seeing that.

I mean, I know GitHub used it in the early days for some of their stuff, but lots of other projects did as well. Suddenly people saw, oh, there’s a really good code editor that covers that table stakes stuff. Now I can build this weird project idea that I had that I haven’t been doing because I don’t want to have to build a whole, you know, fully functional text editor first. That’s just too hard and isn’t the core of the idea. So maybe there’s, if your hypothesis that the canvas is a foundational type of some kind, as you said, how software is just starting to be, then maybe there’s a similar explosion that could be unlocked with the right canvas tool kits.

00:10:52 - Speaker 1: I hope so. I released this thing in November of 2021, so after, I think I started working on it full time in like July, so not too much time. It got pretty popular, the initial usage.

00:11:07 - Speaker 2: You talking about the open source or because I know, I guess I should describe that.

00:11:09 - Speaker 1: While I was developing this, I was posting about it on Twitter a lot. You’re gonna hear me say Twitter a lot, probably during this interview.

00:11:19 - Speaker 2: I think we could have done a build in public episode with you if we already did it with the maker of Canopio Club. Yes, so it’s a similar kind of concept which is showing your work in progress as you go and obviously it’s a very visual domain and yeah, people like that. They like those little bite sized pieces and they like to see the journey.

00:11:39 - Speaker 1: Yeah, I’d started thinking like, this was right when GitHub sponsors had come up, and I guess the media that I was consuming was largely like sponsor driven. Podcasts, YouTube channels, etc. And I was really interested to see if that kind of model could work for programming. Programming does not lend itself well to YouTube streams or videos, even like the educational stuff, I think it can work, but it’s not like a sponsor model that kills, it’s like a course model or something.

00:12:10 - Speaker 2: Yeah, from what I’ve seen that typically people who have programming YouTube channels, then you have an upsell into buy my course, so they put the more basic stuff online, you find it, you watch it, you get to the end and you like the teacher, quote unquote, and then you think, well, I want more, I want the intermediate level stuff, then you go buy their course that’s behind some paywall.

00:12:29 - Speaker 1: Yeah, which is great, and I’ve certainly bought, you know, courses and books as well, and especially if you have an education budget, you know, consider spreading that around, don’t let it go to waste. But I was kind of thinking about the type of content that I could post every day almost, or the or the type of content that required like a very low level of engagement with, like, much lower than by my course, and even like a level of engagement where just clicking like was like the correct.

Appreciation for this type of content.

You know, I’ve seen folks who are making like really amazing involved educational material, not just in programming, but in other stuff too, and it always feels bad where like all I can do is just click a little heart, you know, it’s like, oh man, like this is worth way more than that, kind of feels awkward. So anyway, I kind of made user content, educational content when I worked at a company called Framer. So that was all very involved as well, very kind of produced.

Took a lot of time to make, and I didn’t want to do that anymore.

So, the kind of the place where I settled in terms of like the kind of content that I would use to drive interest around Tealro before I had released it, like, while it was under development, was just like these eight second gifts. These gifts that maybe had like, you know, were at 150% speed, didn’t take very long to make at all, and certainly didn’t take very long to consume and just be done with, and, you know, just clicking that little like button was, what more could you do? Like, it was perfect.

And so I was posting this a lot as I was working on Tealraw and at the same time, I had made the like TealDraw.com a kind of sponsorware. So the only way to access this was to be a GitHub sponsor. And there really wasn’t any like floor for that. You could give me a $1 and, you know, have full access. It wasn’t even like a $1 a month. It was just like, give me something and you can come in. And that worked pretty well. I got like a couple 100 sponsors, which certainly wasn’t enough for like a tech salary or anything like that. But it did show that people were interested in this and it was like a good thing to come back to as I was developing it. I was also a good motivator for making that content.

00:14:46 - Speaker 2: And to come back to that product manager style validation, here’s the next step. People parting with money any amount, the number doesn’t even matter that much, is just such a hugely higher bar and saying they like it, clicking a like button. Telling you they think it’s cool, even using it. Parting with your hard earned cash is just the ultimate measure of validation. So I don’t know if this was intentional, but it seemed you’re following the product management playbook kind of market discovery in this journey.

00:15:18 - Speaker 1: It was not very intentional. Again, I was kind of just playing with this before going to take a job at Adobe, but, you know, I liked making the content, I liked being able to share this before it was ready and get that like feedback from people. And I liked that those people were a little bit motivated as sponsors, and somehow it just really like ended up in a kind of warm place. It felt like there were a lot of people on Tal Draw’s side as I was making this and like wanted to see it do well and wanted to You know, be a part of that. I spent a lot of time asking folks on Twitter like, how should this feature work, you know, this, here’s how it works in sketch, here’s how it works in FIMA, here’s how it works in Miro. What should happen when you try and resize a group of shapes where one of them is rotated? I don’t know, like, does it skew? Does it smash? Does it lock the access or uh aspect ratio and then resize that way, so.

There was a lot of like kind of audience involvement in a sort of a musical sense.

And yeah, that carried right into the release in November, where I made everything open source, like overnight. It had previously been closed source. I’ve removed all the kind of the sponsor walls around Tealra.com. And just said, like, there you go, release, you know, there was no product hunt launch. I didn’t post it on hacker news or anything, but other people did. Enough that I had to ask like product I’m like, can you just stop posting these? Like, I would love to do a launch sometime in the future, but I don’t want to do that. And then it was like at the top of Hacker News for a while, like, for like a majority of the day, it started. Getting a ton of attention on GitHub, like in terms of stars and interest from people there.

My Discord started exploding. It was, it was like a really interesting week as this thing made its way around the internet. And just as like a free diagramming drawing tool that had I poured a lot of like, I guess, attention into the microinteractions to sort of add up to a canvas tool. Yeah, it was fun, and bunch of cool contributions started coming in, and I was thinking, I don’t know, maybe I don’t go to Adobe, maybe this is like a good Maybe I can make this into a little microsas product for myself.

And then what happened was, since it was open source like folks started building with it and I started seeing a lot of interesting projects that started being used or built around Tealra and getting some amazing contributions from people who are kind of using it. And noticing like where it needed to grow and and how it needed to change, and a ton of feature requests and suggestions and will it do this, could you make it do this? Can we contract you to come over and integrate it here or there? And also interest from like investors and other people who are kind of working on similar projects, but there weren’t many. It was a really exciting time.

Eventually, the interest was enough. And I had some conversations with, I guess people whose advice I could take that this might have legs, this idea, kind of like this bet that we’re living in a moment where, for better or for worse, there’s a lot more remote collaboration. People want to do interaction or like collaboration through software more and more that needs to happen somewhere.

The canvas is probably the place where that’s gonna happen.

My bet was these apps, there’s just gonna need to be more of them. It will be impractical for every team that wants to make Miro for project managers or fig jam for doctors who wanna collaborate on annotating images, like, it’ll be impractical for every one of those companies to also build Miro or also build fig jam, which at the moment is sort of, there are no primitives, so, yeah, good luck. Like if you want to do that type of software, like your work cut out for you. If you want to do that kind of software then. Yeah, there’s really nowhere to start except from scratch. So, there’s Tal Draw the startup, that’s just Teal Draw the company Teal Draw Inc and TealDraw GB now also.

00:19:28 - Speaker 2: Well, congratulations, and at the moment is this, the story you told up till that point, it sounded like it was very much a solo effort doing kind of everything yourself, one man show. Is there a team now or is that still to come?

00:19:42 - Speaker 1: Yeah, it was pretty much just me, probably like 99% of the code in the project was just written by me over those couple of months between July and November. I have had amazing contributions from a lot of people on the GitHub repository. And those continue to come in, we have a nice little community growing. But yeah, now I have a team. I’m up to, I just hired my first employee, and I have two other contractors who are taking up different parts of the project. We’re growing it, for sure.

00:20:13 - Speaker 2: Oh, congratulations on your new job as a manager, I guess.

00:20:17 - Speaker 1: Yeah, exactly. I have no experience, but so far so good.

00:20:22 - Speaker 3: So when you open sourced TLDraw, were you at that point specifically positioning it as a platform for making verticalized canvas apps, or was it more open sourcing the app and then people discovered that you can mod it basically and do their own thing?

00:20:41 - Speaker 1: Yeah, so I had picked a kind of a point to chop the app into two, in a way. I was saying, OK, here’s a rendering layer. Here’s a, basically like, put react components on the canvas. I call that the kind of the core layer. And then on top of that, I built a kind of an application that would use that core layer, which implemented things like selection or erasing or different types of shapes. And my guess was that other teams would want to build on that core layer and say, OK, well, you know, Tealro’s cool, but we’re gonna make a different app, so we’re just gonna use the same render, like react Canvas type of render. Um, I was completely wrong. I think there’s maybe one team that ended up doing that. Everyone else just forked Teal Draw.

I picked the wrong place to chop, essentially, that what I thought was unique to the app, to TealDraw. Ended up being more general than I thought.

And so things like the selection tool where you, you can whatever hold shift and click things and now they’re all selected or draw a box in order to collide things and select them that way. Turns out that’s the same for every canvas and I just gotten it right and followed all the conventions that I saw in other apps who had also reimplemented the same sort of logic and whatever an eraser is an eraser is an eraser like.

So the part of the task, once I did decide to keep going on this was to basically start over, like within 6 weeks of releasing, I was like, all right, let’s start over, let’s make this for real. And let’s make that abstraction point that like, where can I chop this? What can I say is general versus what is specific to til draw and move that way up so that it really is, you could get more for free without having to fork it.

00:22:31 - Speaker 3: That’s interesting and that’s super surprising to me. I feel like for these complex problems, it’s basically impossible to design a framework de novo, and people try this all the time, but it very rarely works. Instead, what happens is you have an application that works well, and then basically you copy and paste that 2 or 3 times, and then you look at the discs and the things that aren’t the disks become the framework. And that sounds like it’s basically what you ended up doing with this.

00:22:55 - Speaker 1: Yeah.

00:22:56 - Speaker 2: I think that was a story with rails as well, where there was a bit of they kept writing these kind of crud apps with a model view controller framework, and once they’d done it a few times, it became pretty clear what should be extracted from that and be common to all of them and what would be different.

00:23:12 - Speaker 1: Right. Yeah. Well, even though I did pick the wrong place to chop or the wrong abstraction, it was part of the kind of the DNA of this project from the beginning to be something that other people built on.

And so, once it was evident that like folks just were using Tera the component, which I should say that we did distribute the complete app as something you could put inside of any website. So it was like also something that you might like drop into your video chat app, even if you didn’t want to change how it looked or worked, but you just wanted to have it somewhere other than TDraw.com.

But yeah, once it became evident that folks wanted to say like, Teal jaw is perfect. We also want to add like a shape that represents a person.

You know, and that like looks like an avatar and has like that type of data attached to it. How do we do it? And my only answer was like, smash that fork button and then own this thing forever, because like, I just, it’s not built for that, or start from scratch essentially again, like, maybe not zero, but still very close to it.

So yeah, version 2 is much more kind of Anticipates those kind of stories, and I had no idea folks would want to do that, but looking back, I guess, yeah, now it’s obvious.

00:24:30 - Speaker 2: I see that’s where you are today. I’d love to hear a little bit of the backstory there. You mentioned briefly working at Framer, which is a very interesting product. It’s been through a lot of iterations and itself has its own canvas aspect, and I know you’re actually even relatively new to the programming field, so tell us a little bit about the journey that brought you to Teal Draw.

00:24:48 - Speaker 1: Yeah, I guess, you know, it seems like everyone I talked to has a non-traditional story about how they got into this kind of tech field.

For me, I studied art. I studied, uh, you know, painting for undergrad and then grad at University of Chicago. So I had a tiny bit of like technical experience building like portfolio websites, but that was pretty much it. It wasn’t until I had turned 30, I was now living in Cambridge in England.

Kind of broke, realizing I should probably make some money in my life and have a career that has a little bit more speed to it. So, yeah, I kind of shut down my studio out out near Cambridge and started looking at different ways of using that same creativity and I guess industry, I suppose.

So I started out in design, and then I quickly learned about like prototyping as a place where the tiny little seed kernel of technical skills that I had had from all those WordPress sites that I’d in the Bush administration could be brought over and applied.

This was also like 2017, the idea of like designer who codes was like, even for someone just crashing into the tech scene like was very evidently like hot and so that essentially became my brand or my story as I came in. I was an extremely active user of Framer’s first product, the Framer, now Classic. So typing out coffee script code and making things spin around when you click at them.

And my first couple of jobs here in London were essentially, yeah, like prototyping. Like I was brought in because one way or another, they wanted to be able to build something before they actually brought engineers on to build it. And so this Kind of approach to design and approach to programming of like discovery mode of like, well, we’re not even sure what this needs to be, but let’s start hacking something together. Yeah, that was essentially what I have been doing my whole career and still what I’m doing now.

Ended up working for Framer doing their education, which was an interesting break from actually like shipping designs or shipping prototypes and instead trying to figure out. How do I talk to people about this, or how do I show how to do this type of work? And instead, like, how do I present this to other people? How do I communicate, like, what makes a good prototype? How do you go about that and specifically how do you do that with Framer.

I wasn’t so good at that job. And so afterwards I went back into design and prototyping for a company called Play who’s making a design tool, another design tool for iOS, which surprisingly like, you wouldn’t think that a design tool fits on a phone or on an iPad, but it does. It’s a pretty cool one to check out if you haven’t.

That was a fun opportunity to kind of rethink all of the creative software experience. As a designer, you kind of can’t help but think about the tools that you use because of their software too, and you’re designing software. Play was like an extreme example of being able to rethink features like a layers list or an objects list that haven’t changed since. I don’t know, Adobe Illustrator, and say like, OK, well, how do you do that on the phone? How do you do a properties panel when there’s, you know, the size of an iPhone to work with? That was a blast for me, and it was during that contract when I started doing the open source work around arrows and around a perfect freehand, and then eventually healra too. So, that’s sort of the abridged version of my kind of path from Design and technical design, or increasingly kind of technical aspects of design until now I’m, I guess, not too far away, but they definitely do more programming than design and now managing too, it’s fun.

00:28:49 - Speaker 2: Not to mention it seems like there’s an evolution there from prototypes which are by nature or even by definition throwaway, and then going to something that’s an open source library, other people are building on it, you need API stability. There’s going to be, I mean, so far the project’s pretty new, but you need long term, it’s quite the opposite of that prototyping, so I imagine you’re growing new skills there too as well.

00:29:12 - Speaker 1: Yeah, to be honest, this is also a place where I think the team that I’m building is gonna Do a lot of the heavy lifting because my relationship to software is still very like disposable, kind of like crumple up and and throw in the trash, just do lots and lots and lots of sketches until it resembles what you want. Not to say that Tilra isn’t like solidly built and, you know, we’ve fixed the bugs and it it has good abstractions and solid API and all that. But yeah, definitely there are certain problems that you can’t quite as easily start over with. So, I will be slightly hands off on the on those problems just because different skill set for sure.

But the community side of things, I mean, managing not only the open source, like an engineering project, but also as a community project has been really interesting too. That’s probably closer to the work that I was doing at Framer in terms of education and doing a lot of work with the community and a lot of work on Twitter and uh chats and such of kind of unblocking people as they’ve been working with at that time, Framer now and with Tealro turned out to be decent training for those sort of Open source relations manager, open source project, open source maintainer, culture or role. Luckily we haven’t had any drama with Tera, but it’s still a little bit chaotic and fun.

00:30:36 - Speaker 2: So our topic today is infinite canvas or perhaps infinite canvases.

So I think we’ve explored this a little bit already in talking about in this new world of kind of, yeah, the whiteboard brought into a digital space and made collaborative and that is increasingly feels like a foundational piece of many pieces of software.

So yeah, I’d be curious to, you know, kind of go to the fundamentals here, which is certainly how you Steve or Mark how you define. What is an infinite canvas, and then we should probably also talk about the name, cause that’s an interesting thing. But let’s start with definition. Steve, what do you makes one of these canvases what it is and different from other types of software?

00:31:15 - Speaker 1: Yeah, well, especially on the web, we are used to a kind of a document metaphor that, well, the web is just built for, which is kind of a vertically scrolling infinite page. Infinite canvases are essentially like a different document format. This idea that there are two dimensions or kind of almost 3, you know, you can move left, you can move right, you can move up, down, you can zoom in and out, and that the screen, what you’re looking at kind of represents a camera that’s kind of floating above a surface, and there are things on that surface. And those things can be anything. And in a drawing app, those things might be little lines that you’ve drawn in a visual note taking app or canvas-based note taking app, those would be different notes or different headers or different flags or in a whiteboard, they might include arrows and texts and sticky notes and all that.

The actual things on the canvas don’t matter. It’s mostly about Stuff on the canvas that you can view, or if you’re able to, if you have the correct permissions, then you’re able to directly manipulate.

You can drag things around, you can move them, you can duplicate them or resize them, and that each one of these types of thing on the canvas also sort of has its own rules about how it can be changed. So maybe it’s a video, maybe the aspect ratio of that video can’t change, even though you can’t change the size. Whereas maybe it’s like a rectangle and you can change that aspect ratio or the sizes.

And whatever you wanna call these things on the canvas like shapes or primitives or elements, the thing that you do with an infinite canvas is, you know, to manage these and read them or arrange them or put the canvas into a state where it represents something.

So, obviously, again, like whiteboards are pretty clear, like you wanna do a retro and you’re moving sticky notes around, and maybe voting by stamping things. And the important thing also is that, unlike a document where you’re only seeing a very small part of infinitely scrolling page.

The canvas works really, really well for collaboration. The idea of having multiple people working on the same surface is pretty natural. You just represent them by their cursor, wherever the cursor is, that’s where they are. Um, you can do things like follow people around. I suppose you could do that on a vertically scrolling page, but it might not be as fun. Yeah, so that’s why I think it’s been picked up so readily by, again, like whiteboarding or diagramming or or places where you need to have more than one person editing the same document. It’s not like something with text where, as I’m editing text at the top of the document, all the other text is being pushed down the document.

00:34:00 - Speaker 2: The elements are relatively independent of each other. Yeah.

00:34:03 - Speaker 1: Yeah, exactly. You know, you can have independent experiences on the same document that are just in different places.

And I suppose the idea of place is also a big part of this, and potentially a big part of this.

The idea of near and far, of close and distance, or like it quickly drifts into like video game territory of your cursor is your avatar, or maybe you have an avatar and you’re moving that avatar around this top down view of an office or a space and jumping into video calls with people close to you. So yeah, that infinite canvas, that’s kind of a long definition of how I see it. It’s just a found an infinite 2D plane and you are a little camera represented by a cursor. Moving around, moving over that surface. Mark, what do you think?

00:34:52 - Speaker 3: Yeah, I have a pretty similar take. I’ve often called these multimedia canvases, cause I think there are two dimensions going on.

So one is the canvasness, which is at least 2, sometimes 3 dimensions and the freedom and flexibility to place content items wherever you want, like you were saying with relative independence.

The second axis is multimedia, so Which types of media the digital document accommodates.

When you lay it out like this, it’s interesting because this actually captures the whole universe of digital document editors.

So for example, a classic plain text editor is at the sort of bottom left of this graph where it’s very limited to one content type and it’s very linear, a spreadsheet in contrast. Has pretty limited content types. It’s basically text and numbers, maybe a little bit of color, but it’s quite high on the canvasness.

You have this flexibility and freedom to place things and by the way, as we’ve talked about in this podcast, that’s something people love about spreadsheets. It’s not just a calculator, it’s just a place where you can put stuff, people like that.

Then you can go all the way out to the top right where I would put like muse and TL draw where you have a full 2D canvas and it’s highly multimedia. You have handwriting, text, images, videos, links, whatever.

Or you could kind of go back down and say we want lots of different media types, but we don’t have the full flexibility. So I would put notion in this category, for example, of you basically have whatever media type you want, and there’s, there’s a little bit of flexibility, but it’s pretty much a linearized document.

So in this view, the multimedia canvas is simply the fully generalized final form of a digital document.

00:36:38 - Speaker 1: It’s good to be here at the end, yeah.

00:36:40 - Speaker 3: And all the others are sort of specializations of that space.

00:36:45 - Speaker 2: One element you touched on there, Steve, which also I think fits in with the multimedia side as well as you talked about the elements, you know, we call them cards and news just because I think that works for us visually and particularly with the touch screen. It feels like an index card moving around on a desk or something, but yeah, the elements have a certain sameness and I think this does go back to Illustrator, which in many ways was the original Infinite canvas to my mind, but you know, maybe sketch and. FigMA and Framer kind of modernized that a little bit, but when you lay down a bit of text or add an image or add a rectangle in, for example, sketch, you can click on and select each one of them. You resize them the same way as you said there might be slightly different rules around resizing.

There may be snapping or other things, but in the end, You can do the same things with all of them, and I think that’s really important and it’s actually something that to me comes actually from file systems.

This is what’s great about files. It doesn’t matter what’s in the file. You can always delete it the same way. You can copy it the same way. You can put it in a folder the same way, you know, inspected size, that sort of thing. There’s this uniform container. And then over the years, files have gotten more capable and contained more and more different types of things, including, you know, things like video that the original creators of the file system probably couldn’t have even pictured being possible on a computer, but because it’s this general purpose container and as a user, I feel that gives me a lot of agency and power because even if I don’t know the specific type, I know exactly how to manipulate it.

00:38:16 - Speaker 3: This is reminding me that there’s potentially some secondary axis here. So one is collaboration, which we’ve mentioned, and we almost take that for granted now when we think about modern software, but you know that is a separable axis. The other is this notion of inline editing, which I think is actually pretty essential to what we think of as a modern multimedia canvas app, in contrast, you know, with the typical file manager like Finder, that’s actually very high on multimedia and quite high on. 2D flexibility, especially on your desktop, but you don’t have the inline editing, so it it feels like a totally different experience.

00:38:50 - Speaker 1: Yeah, this ability to not only move, delete, organize, copy, hold alt to clone, to directly manipulate content, but also It’s not only about making choices about where a thing is, or its relationship to other things, but it’s the place also to make stuff and to edit it directly.

And yeah, one of the bigger kind of challenges with tealra was deciding how to allow interactions within shapes versus interactions with shapes.

So, for example, you have a video, you can click on the video and drag it around. But you might also want to like pause the video or change the time of the video, and thinking about, OK, well, how do you transition from a shape that is, I call them shapes, whatever tail drops I’m gonna keep using that word, too hard to deprogram a shape that is Acting like a shape versus in a shape that is acting like a video, and that you can interact with with like a video.

And yeah, it turns out that there’s a pretty good rules around that, like how to make it consistent, which works just as well for like text as it does for videos, as it does for, I think I have a code sandbox, you know, running on the canvas and teal draw in one of my example projects. The joy of using a kind of a web-based canvas of rendering stuff using the web, although that sounds like such a bad idea to render things in HTML and CSS, but it really does give you the ability to just put anything that can be in a browser on the canvas and interact with it.

00:40:29 - Speaker 3: And this also brings us to the why now and why is this hard. Like if this is the fully generalized form of digital documents, why didn’t software start this way? I think a big part of it is just that it’s hard. Like we talked about how it’s hard to write a text editor, but what if you now need to write a text editor, an image editor, a video editor, and audio editor, and they all need to be in the same thing, you know, it’s, it’s quite difficult.

And then on top of that, you got to actually render all the stuff and make it manipulable and fast and responsive. It’s just quite difficult. I think that’s a big reason why.

We haven’t seen it until relatively recently.

I also think there is an element of people just, they don’t fully realize the expansive possibilities of the software, perhaps when they’re starting from a very limited. World.

But OK, I constantly bring up this analogy of like a woodworking shop. Could you imagine if you had like a woodworking district and you had a shop just for like your chisel work and then a shop for your saw work and then a shop for your standing work and like you had to take this, you know, quote unquote file of a project and bringing it across to these different buildings. That’s kind of the world that we live in now, and you can’t even look at two things at the same time, like you’re standing and your chiseling work. That’s kind of the world that we’ve been living in with respect to software for a long time. And yeah, it takes some work to bring all those things into one. Workshop and to learn all the tools and to keep them all maintained and stuff, but that’s what you really want as a creator.

00:41:43 - Speaker 1: Yeah, especially in the browser, it’s like. Every 10 years, I’m told that you just have to forget what you knew about like what you can do and what you can’t do just because it just gets that much better.

Certainly when I was Poking around with WordPress websites in college, the web was not a place where you could make a kind of a dumb driven canvas and make it like fast and good and perform it and do all the things that you need to do with pressure and multi-touch and all that.

But it is possible today, and I think that when a platform is mature enough, you know, for you to have like an anything app, kind of a place where you can just put anything. OK, cool. Videos, yeah, put it there. Text, we’ll put it there. And that’s always sort of been the promise of the browser. It’s just been really constrained to this document format that is increasingly showing its limits.

I think one of the newer APIs that I was looking at from Google, you know, involved like transitions, like kind of iOS style transitions between things, you know, that absolutely explodes the notion of the web page as a page and hopefully Tera also kind of demonstrates that this technology could be used.

By the way, like, the idea of a rack driven canvas, not to keep coming back to framer, but That’s where I saw that this was possible because their, their canvas is driven by react and is driven by the dumb and like surprised me with the fact that it worked and that it could be made secure and all that. So, I don’t think I would have come up with this project without having seen it been done by people more capable than me than considered that as a possibility. It’s like, hey man, you could put anything on here. Like, why aren’t we?

00:43:26 - Speaker 3: This was an important data point for me in believing 4 or 5 years ago that these multimedia canvases were going to be really important because anywhere you had anything that was like multimedia canvas, if you squint, people loved it and they were using it for all kinds of stuff. I mentioned the example of spreadsheets. Another example is using tools like PowerPoint or drawing programs to do like whiteboarding basically. We’ve also seen this more recently with FigMA. FIMA is obviously a design tool, but people will use it for a personal note taking just because they really like the ability to put different stuff on a canvas.

00:43:56 - Speaker 1: Yeah, I think one of my first creative software experiences was with Hypercard, way back in the day. I think it was still Hypercard, like Apple Hypercard or not macromedia or Flash or anything like that, but yeah, there’s this idea of like, that is not an infinite canvas, that’s just a glorified slideshow.

But it’s still like I’m like, man, you can make games with this, you can make presentations for this, you can make You know, animations, if you just click next fast enough.

Yeah, um. A little bit of Like giving users a little bit of room to run, you know, it’s always like the best, just because, yeah, suddenly you have Doom running in Microsoft Excel.

Yeah, and I think with TLDraw, I mean, folks have made a slideshow app, like a slide editor app with TLDraw, which apparently didn’t take too much work cause it already had like pages and you’re just have a UI for moving between pages and I’ve seen it used to do video annotation, like you pause a video at a certain time and just overlay like teal draw and be able to edit, like, you know, this guy’s gonna run over here or whatever, and it’s been used for like wikis.

One of my favorite stories about this actually is that there are two products that compete with each other. They’re both for Dungeons and Dragons game masters or Dungeon masters. And when you’re telling or running like a story campaign or a story world game, you have a bunch of locations and people and, you know, items and all this stuff and they might have relationships to each other, etc. And so there are two pieces of software, one is called WorldAnvil, one is called Legend Keeper, and they both are essentially wikis, custom wikis for people who are running these games. And Legend Keeper shipped a whiteboard view of their like wiki, essentially. Based on TalDraw, like 6 weeks after I shipped TealDraw, like open source it, like immediately, they seemed to be like immediately, oh cool, this thing that we wanted to do for like 2 years, like, let’s just do it. Like we could just nail this, and they did. And then about 6 months later, World Envil released their canvas, their whiteboard view of their wiki, which was also based on Tealra. And I had been, I’d been telling people I’m like, someday there’s gonna be a product. Or two products that are competing with each other, similar features, and the thing that they’re not gonna be competing over is like how well the eraser works or how well the select tool works. They’re gonna be competing at that higher level of, yeah, like the features that they built that are unique to their products because both of them are gonna be using TealDraw. And then it happened 6 months in, it happened that I was like able to point to both of them and I love that. They’re both wonderful products and then their communities are always posting. Like I kind of lurk on their Discord channels just to see people’s like whiteboards, and yeah, it’s like 200 individual characters, you know, in a big family tree structure or like alliances between things. Yeah, definitely not what I expected when I was working on Tealdrop, but that’s the whole point, right? It’s like to see where people take it.

00:47:06 - Speaker 2: That’s great and I like the Dungeon master and world building aspect. We have some new customers who do a similar thing for their sort of dungeon mastering, but it is in a way the purest form of knowledge management.

Yeah, you have this complex world, you need to keep track of stories, characters, yes, it’s all made up.

There’s a version of this for sort of fictional worlds as well, right? Like, yeah, if you work on some franchise like Star Trek or whatever. to keep some shared knowledge base of all the canonical everything that has ever happened, timeline and characters and rules for the universe and things like that, but yes, it is a mix of obviously textual, kind of more linked wiki knowledge graph style stuff.

But obviously diagrams, maps, drawings, all of that can and should be a part of it, but traditionally the digital tools that we use tend much, much more towards those that text and that linear top to bottom document, and it’s not even top to bottom, it’s really about the inline. It’s about the text that flows left to right and it wraps when it hits the edge of the screen and it keeps going down until you get to the bottom and any multi.

The media you add tends to kind of float awkwardly as sort of a big character or something, but I think when you look at something like the Dungeon Master use case which so purely and in this made up realm captures the complexity of what you can do with computers in terms of tracking knowledge and what you might want to do, and quickly you see that text is a big part of the story, but it’s not the only part.

00:48:38 - Speaker 3: Yeah, and I think related to this idea of constrictive media types and layouts is the notion of premeditated workflows versus totally in the control of the user workflows.

So typically with software, you have workflows that are Designed by a product manager or whatever, and they write down the software and that’s that. So you can imagine for world building, you have a database table that’s like characters and database table that’s equipment and you know, if you have something that doesn’t have a common database, well too bad, you know, email the product manager maybe able to add it in a year.

But people really like the ability to craft the software to their use cases and motivations. I think this is a big draw of The multimedia canvas, you can just do whatever you want with it.

You know, if you have a different way that you think about equipment or characters or whatever, you just put it on the canvas and and do whatever you want.

And I think that’s an important aspect.

But the other thing here is you can go down a layer and change the actual programming, which is one of the things that sounds like it is really exciting with TLDraw. And again, people, when they just give them a little kernel of power and capability and you meet that. With some motivation, people do all kinds of stuff. I’m reminded of the Half-Life game engine. I don’t know if you guys know the story, but the game itself was great. It was very successful. But this game engine spawned off all kinds of stuff, you know, that critically, the original creators did not anticipate, design, approve, or even know about, right? Just someone else, they took this kernel of power and made something new. I think that flow was so great.

00:50:05 - Speaker 1: Yeah, absolutely. One of the opportunities of having like a multimedia canvas is the ability to extend it semantically, almost, and you see some of this discourse around like the idea of notion-based editors, block-based editors, right? Is that like text is a primitive, images are primitive, but you start to kind of wrap these things up into more specific domain specific or just project specific and more meaningful blocks, right? And those things can interact in ways that maybe have to do with like what they are. So in the kind of the Dungeon Master example with something like Legend Keeper, the things on the canvas that they added were parts of your wiki that you can put on the canvas. And so the character on the canvas, you know, you can drag out a character from your wiki and put it on the canvas, is not just an image of that character, but it’s sort of is a representation of it. And It could be that dragging a character onto another character might produce a different outcome, might suggest a different user intent than dragging a character onto a location, right? Now suddenly you’re able to extend this surface metaphor into something where those same basic interactions, those same basic direct manipulation actions can be like meaningful in a way and produce different outcomes based on what you’re working with. Um, it’s not just text, it’s not just images. Now, we have people, we have places, we have things, and they might have their own rules around them.

00:51:34 - Speaker 3: Yeah, I think this idea of extending the content types in the canvas is super cool. We explored this a little bit back in the day with the prototypes that led to Muse in the Ink & Switch research lab, and I remember a couple that were really fun.

One was a person. Which is represented as like an avatar and a name, but it was so useful to be able to have someone on the board and put them next to tasks or put people in a group, or, you know, put your current work under your picture. It’s like a great primitive, in other words, maps and locations, something that actually people use all the time for planning purposes.

But I think there’s an important sell to here around extensibility. So there’s one thing you could do, which is draw out the notion of a box and say, we as the creators of this program could put different things in this box, you know, notion could put different things in their block abstraction. And just that is useful because then you can manipulate these in standard ways and link to them and so on. But then users being able to extend it is so fun and powerful. I remember with our prototypes, because this was back when we were doing the prototypes in JavaScript and people could just kind of write the stuff like a react component. And it was people were so excited to be able to write their own things, you know, things that were too specific and niche for the central, you know, controllers of the framework to ever worry about or even think of, but people felt so empowered to write their own components. This brings me to a problem that we’ve been noodling on for several years and and to my mind it’s still a critical open question. So when you’re trying to build these end user extensible digital document systems, there’s a few deerrata that you want. OK, you want it to be very fast, you want it to be safe in the sense of end users aren’t gonna be injecting wild stuff, you know, and other users' data or something. You want it to be approachable cause you want end users to be able to actually use this thing. You don’t want to give them some like, you know, assemble or guide and say go have fun. And ideally, I think you want the extensions to not feel like a totally different world, like some limited, slower, neutered, you know, subset of the platform. You really want to feel like the extensions and the platform itself are like written in the same way. And in fact, if an extension does well, it can be promoted into the platform. Or if a piece of the platform is, you know, not finding a lot of use that can be sliced off as an extension. And I just think it’s very hard to get all these things at the same time and it’s not clear to me how you do it. So for example, C++ can be very fast and It can be safe. Maybe Rust would be a better example of something that’s very fast and safe. But then telling end users to write their components in Rust, that’s a little bit harsh. And JavaScript in our experience can be very approachable, it’s very flexible, but it can be very hard to make fast, especially at the 120 FPS level that we’re targeting from use. So I’m just kind of curious if they that problem statement kind of resonates with you, and if so, how you’ve been approaching that.

00:54:25 - Speaker 1: Yeah, I think that’s a really good summary of the problem of extensions.

Um, which some of that is gonna be true for any products, right? Like it’s true for a text editor as for a video conferencing infrastructure. I don’t know, whatever.

In the case of Tealra. I’ve addressed this in kind of three design decisions. One is to use React for the canvas again, because I think the development story there makes up for the performance story. And the performance story is pretty good. It’s never gonna be as good as something like custom webGL canvas that was written in a systems language, right? That’s not gonna be possible. But on the other hand, uh, you can put things in there that you just can’t put in anything else. You can put a mapbox map or you could put a, you know, YouTube video, like as we’ve talked about, and you can do that, you can offer those things fairly easily. So, in the case of Tealra, all the shapes that you and I’m kind of referencing this next version that’s gonna be launching soon, all the shapes that you see. When you look at teardrop when it loads up, those are essentially extensions. Those are all custom. We wrote the user facing app kind of at that same level as you would write your custom plug-ins and your custom things. And I was pretty good about not cheating, and of course it’s easy cause we can change that lower level, you know, that core level in order to facilitate things that don’t work yet, but um all the shapes are custom shapes or plug-ins. And then the shapes themselves, these sort of like plug-in shapes. are all based on like uh primitives that the lower levels do share. So, for example, you have something like a box, right? A box rotates in a certain way, it resizes in a certain way, it hit tests against like points and lines in a certain way, and all of those things are different than if it was like a pencil shape, like a drawn line. All those answers are different. However, they’re all the same. Every box is gonna hit test pretty much the same way as every other box. Every line is gonna hit test the same way as every other line. So when you’re creating these custom shapes, when you’re creating these extensions, you’re able to just sort of inherit all that functionality from kind of the base, right? You say this is a box, all right, job done. This is its model. It has a width and height just like a normal box, but it also has a latitude, longitude and a camera zoom. And then the next question is like, well, what does it look like? And you say, well, it looks like a map box component, and it uses that data from the model, and that’s pretty much all the code you have to write, because all the other parts of that box in terms of its behavior and such are just their default, right? If you wanted to make something crazy and unique or if you wanted to make something that was very different than all of the primitives that we do share, things like whatever boxes, lines, polygons. Then you can also access those same primitives, those like lower level primitives to say, OK, well, here’s my squiggle. It has 3 ups and downs. Here’s it’s like outline that you can use for hit testing, and here’s how it resizes and all that. But most of the time, the custom shapes that we’ve written are just boxes, mostly just boxes with react components inside of them. So that’s the authoring experience, or the like developing story there, which itself is a developing story as we try and push like what you can do with these type of shapes. Then separately, it’s like, how do you do that, especially in a multiplayer situation, how do you allow multiple people to be using their own custom shapes, especially if they’re sort of end user credable. At the moment, the answer is, well, you don’t. The way that I’m thinking about this is deferring to sort of the implementer level, is that TealDraw as a SAS product might someday have its own extension markets like you might find in Figma where you can download stuff. But in the short term, this is primarily infrastructure tool for other teams to create apps with. Those folks would be the ones defining, you know, OK, well, here’s my person place thing shape. Those folks would be also guaranteeing that everyone who’s using the app has access to those same shapes. And so it wouldn’t really be like end user developer who’s making those extensions.

00:58:38 - Speaker 3: Yeah, so each instance of the TLDraw derived app chooses its extension ecosystem and curates those and makes sure that they’re safe, and it’s not every individual Joe, you know, injecting JavaScript into everyone else’s computer.

00:58:52 - Speaker 1: Yeah, exactly. There might be a future where we want to solve those types of problems or like create those types of marketplaces. And there are a ton of interesting questions about like, well, how do you distribute those shapes in like a multiplayer situation with people who haven’t installed the plug-in. What happens if you copy your squiggle shape from one project into a different project that doesn’t have that squiggle shape? Those are all questions for later down the road. I think the primary thing that I’m going for with this version is just giving other developers the tools that they need in order to build the experience that they want for their users to have that involves this sort of canvas.

00:59:27 - Speaker 3: Yeah, and I think that’s a very reasonable and practical set of trade-offs, you know, react, curated by the instance, makes a lot of sense. It is fun to think about the fully general world.

00:59:37 - Speaker 1: Oh, totally, yeah.

00:59:39 - Speaker 3: Of arbitrary extensions.

00:59:40 - Speaker 1: I’ll get there eventually. There’ll be some cool streaming JavaScript modules, you know, that are pulled in at runtime in order to, you know, we’ll get there, don’t worry. happy not to be thinking about that yet.

00:59:52 - Speaker 2: I think of the time, certainly.

00:59:54 - Speaker 3: And your experience with React is that it’s fast enough. My recollection, so this was a few years ago, but my recollection was that React or React like systems on the web for this canvas use case was Not fast enough.

01:00:09 - Speaker 3: More like was very close. It was right on the line in the sense that if you kind of did anything weird or made a little performance mistake, it’s very easy to throw the flags, not meeting even 60 FPS. I don’t think we ever could try to get to 120 FPS because the browsers on the iPads don’t support 120 FPS as far as I know.

01:00:34 - Speaker 1: Yeah, that’s true, that’s true, but it’s like plausible for 60.

01:00:37 - Speaker 1: Yeah, I think Chrome goes up to 120, and I mean you could try out on Teal Draw, even though the current Tealraw.com is sort of the first version, and that has only like a little bit of the optimizations that we’ve since done with the next version. It’s primarily a memory management issue because those big garbage collects is what gets you. And it’s also about having a lot of control over what is rendering, of course, what should react, recognized as a render. And then managing paint.

Paint is a big thing with canvases also, which the browser can do a lot on its own, and there are some CSS optimizations that you can really push the rendering engine to do as little as possible and to only do it when you want to do it.

01:01:21 - Speaker 3: The memories are starting to come back. I’m recalling, if you use certain CSS incantations, it invalidates a whole suite of repaint optimizations, so you get totally hosed, so you got like, you know, the specific CSS, you know, magic words to say and to avoid.

01:01:36 - Speaker 1: Yes. And they’re not all the same between browsers either. Those magic words, for example, like uh just as a quick example, I think in Chrome. They perform their transform operations, they scale, and then they translate or something. They scale last, essentially. In Safari, they scale first, or maybe it’s the other way around. No, no, Chrome, they scale first, Safari they scale last. So essentially that if you do some of these optimizations that sort of skip the paint and say like, don’t repaint that, you’ve already repainted it. Suddenly everything’s blurry if you’re zoomed in because they’ve scaled it second. Yeah.

01:02:14 - Speaker 3: Another way to articulate this might be the react model in the abstract. If it correctly maps down to GPU operations, is, it’s great and it’s totally suitable.

And the challenge is Getting that mapping to happen correctly through your use of react and CSS incantations and so on. It’s a little bit subtle because that’s a little bit of a black box, you know, there’s several layers there, and like I said, if you make a mistake, you end up repainting the whole screen and you’re out of luck.

I also think just taking a step back from the very down in the weeds implementation details, there is a challenge with canvases and the react style cause it’s less obvious, I think, what needs to be recalculated.

So in a traditional document. You have just take a very simple case, like you have a sequence of rectangles that gets rendered, and when you change, you know, a div, for example, you need to re-render the div and then put that back in the tree, but you kind of know that the rest of the stuff hasn’t changed, at least in a simple case.

Right, because it’s a series of non-overlapping rectangles.

But when you have cards on a canvas, for example, there’s all these implications around, you know, intersections and hit testing and overlapping, and that’s just for rectangular cards, you know, add ink and erasers and it’s a whole other thing. So, I’m just kind of recalling a few years ago, the challenges that we faced with using the reactor model on a canvas. It’s not easy.

01:03:31 - Speaker 1: It’s not easy. Having done it a couple of times and having kind of rebuilt that engine, that rendering layer a couple of times.

One of the tricks was to keep it as flat as possible. So, for example, HTML allows elements to be placed inside of other elements, and they’re all can be positioned relative to one another.

Don’t do that. If you’re if you’re building a canvas, you want to be managing those positions yourself, and you want it to not be pushing that type of change through React, right? So, if a shape has two shapes as its children, even though those shapes are being positioned relative to the parents, and if you move the parent around, that the children move around too, in the react model, those are siblings. They have transforms that are being recalculated automatically based on their dependent data, which is like the parents transform.

And so that takes a little bit of the lifting off of React, to say like, don’t worry, the thing that you are gonna take care of, we will take care of, and then to use a ton of essentially observables in order to kind of Take the wheel in terms of the data. So, You know, nothing has props, like, OK, that’s too much work for React to do. We’ll take care of it using observables and we’ll update things when they need to update. So, there is a lot going on in Tealraw where like, I think we had someone on the Discord being like, hey, I’m kind of new to React, and this looks like a really fun project, and I’m like, this is the wrong project for someone who’s new to react because we’re actively short circuiting like a bunch of the things that React normally does in order to be fast. In this type of an app instead.

01:05:11 - Speaker 3: And I would imagine that the programming within a component looks more standard. It’s like a sort of regular, but then what you basically want is you want react to render each card, and then to take all of those and give them to the GPU to composite because that can be done super fast. So as long as you don’t ask, you know, react to the equivalent to like recalculate and re-render this whole tree instead of just making individual cards you can posit down, it should be pretty good.

01:05:39 - Speaker 1: Yeah, and that’s pretty much a summary of like what how we do it is that if nothing’s changing with relation to with a card shape element, we almost want it to be treated as a texture.

In fact, we do want it to be treated as a texture. Don’t think about it, don’t re-render it, don’t repaint it. But then, you know, you have something like someone zooms in and suddenly you have to repaint everything and have to, but at least you don’t have to re-render it. You just have to, you know, repaint it. And there’s a lot of these days you can use CSS to avoid a certain type of data flow that otherwise would have had to be pushed into all those components. So, for example, maybe it’s the width of the Indicator, when you’re like hovering over something, right? You want that width of that indicator to be based on the zoom, so that it’s always exactly one pixel on the screen. And if you’re zooming in, then you’re actually scaling this canvas using CSS, but you don’t want that line to also scale. You want it to like counter scale it against that, right? In the past, you would have had to push that zoom level down into that box component and say like, OK, well, now the zoom has changed this, you need to adjust the width of your whatever. No longer the case, you can use CSS variables and calculations and such in order to get around that. And so now suddenly, you are still pushing the data around, but it’s happening in CSS rather than in JavaScript land, and so that’s a whole class of things that no longer has to Deal with, did my data change or whatever. And so, especially the things that happen like moment to moment, like dragging interactions, like selection and hovering and all that, I try really really hard to at least keep those things isolated, if I can’t keep it out of JavaScript completely. Yeah, I’ll probably end up having to hire like a X Mozilla browser CSS Wizard to optimize all the all the browser stuff because it is like, we’re not writing a render, we’re not writing a systems level like WebGL render. Instead, the browser is our render and so we need to have as much knowledge about how that thing works as we would if we were working with a lower level architecture. So, if you’re interested in that, My email address is steve@tealtra.com and uh get at me. I’d love to chat.

01:08:04 - Speaker 2: If I can transition us out of the technical deep dive, one thing I’d be curious to get both your take on before we start winding to a close is the name infinite canvas, whether that’s a category or a component, and it’s one that I have some mixed feelings about, because on one hand, Mark uses the term multimedia canvas more commonly we called Muse a spatial canvas and kind of our 1.0 positioning.

And I also like the term open canvas, we actually do use that a bit on our website talking about kind of the interaction model, I guess, or the idea that it’s more about the flexibility and the fluidity rather than kind of infinite sounds more like how big it is.

But one thing that I did learn is that that name actually already has a lot of traction, if that’s the right word for it.

We get lots of reviews on the App Store, people tend to just organically reach for that term they have since the beginning, and furthermore, that even experimenting with App Store search ads, I discovered that actually infinite canvas is a very frequently searched term.

I’ve learned over the years in trying to define new spaces for software and technology that some name that just seems to stick with people, whether it’s Tool for Thought or Infinite canvas or something else, it’s sort of like not worth your while to fight it, so I’ve got my quibbles with it. I’m not sure it’s the best name or the one that I would pick, but it does seem to be the one that has some legs. I’m curious how you both see that.

01:09:34 - Speaker 1: Yeah, not to mention that in the browser, we have something called Canvas, which is a specific technology which is images, offering images through JavaScript and like tealras as we talked about, it’s like emphatically not that. And so I’m always presenting it as like, it’s an infinite canvas that’s not using Canvas like and it’s immediately I’ve, I’ve gone overboard.

01:09:55 - Speaker 2: I mean, canvas is just a lovely word, right?

01:09:57 - Speaker 1: So, yeah, of course. And I like the potential for infinite to not only imply.

01:10:10 - Speaker 2: How big it is, but also like what you can do with it, or like, that it has that other sort of depth, but yeah, possibilities, which definitely that interpretation is less, how big is it or can you zoom in an infinite amount, but more about that it has a lot of possibilities, you can put anything on it, you have a lot of freedom in how you manipulate it.

01:10:24 - Speaker 3: Yeah, I think about it very similarly where, again, to me, the important parts are the freedom and the flexibility and openness, so I might use freeform canvas, but also to me canvas kind of implies free form, you know, it’s a blank cheat and do whatever you want it’s kind of the whole point. So that’s why I often use multimedia Canvas, but I also agree that it’s probably not worth fighting too hard, you know, if everyone’s already using those words, just say it back at them.

01:10:50 - Speaker 1: Yeah, I would be curious to look at that data about what other, what other terms those people are looking for, because the other risk with canvas is, and again, not made any easier by the fact that I picked a name that has draw in it, but that like canvas is associated with like a drawing or painting kind of like context. And so I’d be curious if folks that you’ve seen were searching for like infinite canvas, plus art, plus drawing, plus like Apple pencil. Rather than infant canvas plus productivity plus whatever, collaboration, etc.

01:11:24 - Speaker 3: This does bring up the very interesting question of what is the appropriate physical analog, and like you’re saying, it’s not exactly the canvas because canvas, while it is free and open, it does connote more drawing and art. To me, the analogy was always a desk, it’s a place where you put down different content and work on it, but I don’t think anyone is searching in the app store for Infinite desktop

01:11:43 - Speaker 2: desktop as a metaphor has already been used.

01:11:46 - Speaker 3: Yeah, I got a little bit more traction with whiteboard because people do use a whiteboard in that way or a pin board cork board, but it is an interesting point that there isn’t quite an obvious physical analogy that people tend to reach for.

01:11:59 - Speaker 1: Yeah, Pinboard seems to lack the direct manipulation aspect of whatever this type of thing is now that we’re calling it. I like whiteboard too, although whiteboard also It seems like a fairly narrow subset. For example, like Scalira uses a virtual whiteboard as it’s kind of like description of itself, and that’s really good, but it also, you don’t expect to have like what you’re describing of like multimedia types on a virtual whiteboard. So an infinite white open pin board is really what we’re going for here, just doesn’t roll off the tongue.

01:12:37 - Speaker 2: A whiteboard is an interesting term. Digital whiteboard is one that I’ve seen Miro and others use to describe themselves, and I think it’s not bad.

It feels a little trivializing or it seems to miss the grandeur and the potential that I think this space has because a physical whiteboard is pretty limited not only in size, but really what can do with it, which is just draw in very limited ways in a few colors, whereas again I think the file system is more of a source of inspiration to me, something that is extremely flexible, powerful, searchable, has a lot of depth, has a lot of extensibility, has programmability.

So I think whiteboard isn’t bad, but maybe an infinite canvas indeed is better.

01:13:19 - Speaker 1: I was curious, when you were using the kind of term spatial, like, I use spatial canvas, I think when I was first presenting this idea to investors, and that seemed to click with that crowd, because they probably heard it from other people who are pitching these type of apps. But yeah, does that have legs? Did people understand what Muse was when you were calling it like a spatial app or a special UI?

01:13:43 - Speaker 2: Yeah, I think so. I think it wasn’t very accessible, so I think folks who are, you know, read the Humane interface by Jeff Raskin or know about Zoomy UIs or I don’t know, just have a very deep interest in the topic or maybe that’s more likely for them to understand, but for a more Not even casual user, but just a regular professional user that is looking for a tool to solve their problem. I don’t think it meant a lot, it didn’t seem to resonate, didn’t seem to strike a chord necessarily. I like it, and I’ve even seen others use it as well, but it feels like that’s one’s less likely to really get the traction.

01:14:19 - Speaker 1: I guess zooy won’t take off either, for the same reasons, yeah.

01:14:22 - Speaker 2: Probably not, yeah. ZUI, yeah.

01:14:26 - Speaker 3: This is very interesting to me that there’s two ways we’re defining fitness of the name. One is, for example, if you’ve used uses a lot, does it seem to accurately describe what it does, but the other is, what are people typing in the app store when they want something like this, and these things are actually quite different, and I think the latter is harder.

01:14:47 - Speaker 1: Yeah, I mean, you always want to try and meet people halfway, but it also, I don’t know, it’s been fun to watch what people’s expectations are with Tealraw, like when they come to it, and like, due to some poor user interface choices on my side, like, it’s not obvious that this is a multiplayer app and that you can like, use it with other people, and so I’ve definitely been like on Twitter and people are like, man, this is so good, but I wish that it was multiplayer, and I’m like, oh no, it is, it is, it is, and I know that I’m only meeting like the one out of the 100 people or 1000 maybe who decided to actually tweet about it, rather than just being like, oh man, whatever, it’s not multiplayer like Miro is. So I think getting some of the naming can also work to kind of signal those type of things, like, what can you do with this? What is it for? And Yeah, like if it was called a multiplayer whiteboard, like right off the bat, you know, chances are people would look a little harder for that button to say like, OK, open a new multiplayer page.

01:15:45 - Speaker 2: Notably, Mirro was originally called Real Time board, which I think sort of captures, maybe not the most elegant name, but it does capture that feeling, right? And then later they rebranded once maybe they were big enough that they didn’t need the product’s name to be so descriptive, so. I don’t know, but we look forward to an upcoming rebrand of TL draw to TL collabra or TL I use realtime board apparently it’s free.

01:16:07 - Speaker 1: There you go, waiting, waiting for it. I, I did always feel bad for Mural, M U R A L, which is another direct competitor of Miro, and then they had that name first and then Miro changed their name to Miro, which is almost like identical pronunciation. Yeah, surprised it worked, but yeah, I hope no one else does it with Seal draw, CL draw or something like that.

01:16:36 - Speaker 3: Well, regardless of what we end up calling this, I think the good news is it’s a very powerful category of software, especially over the past 5 years. I think we’ve really seen it come to the forefront with products like Sigma, and now with tools like TLDraw, we could see a whole another blossoming of verticalized instances of this pattern. So I think it’s really cool to be a part of that evolution in software.

01:17:03 - Speaker 1: Yeah, for sure. I really just can’t wait to see a bit like what you were describing with text editors, you know, like, once we had a good text editor on the web, something that you could use to build other products, you know, we saw a ton of really interesting projects that just wouldn’t have been built, because like the same team that wanted to build code sandbox might not have. Wanted to build Monaco, right? Or might not have also wanted to build the text editor that you would need to build code sandbox. Whatever else happens with Tera, I really, really hope that it lowers kind of the barrier of entry to making this type of app, that if there are bad ideas that involve the canvas, that suddenly we’ll see bad ideas that involve the canvas because there’s not 2 years of engineering time in order to get that bad idea out into the world. And I’m more than happy to do that 2 years of engineering time cause I Like this stuff, but I’m looking forward to seeing what people do with it.

01:18:01 - Speaker 2: Well, let’s wrap it there. Thanks everyone for listening. If you have feedback, write us on Twitter at museAppHQ or via email, hello at museApp.com. And Steve, I want to thank you for helping inspire and push forward the infant canvas revolution that hopefully we’re all going to be a part of.

01:18:20 - Speaker 1: Thank you. I should say, if you want to kind of follow some of my journey here, you can follow me on Twitter. I’m at Steve Rui OK. And yeah, if you want to try out TealDraw itself, it’s at tealdraw.com. It is multiplayer. Don’t let the UI fool you.