Transcript

00:00:00 - Speaker 1: As software developers, maybe we go towards building a GUI with specialized inputs and forms and controls too soon, because it’s so much easier to explain to the computer what the user means if they use a specialized input tool like a button check box and so on. But if that weren’t the case, if it’s easier for the computer to understand what you mean as you’re typing in your note, then suddenly text input is the primary thing.

00:00:32 - Speaker 2: Hello and welcome to Meta Muse. Muse is a tool for deep work on iPad and Mac. But this podcast isn’t about Muse product, it’s about the small team, the big ideas behind it. I’m Adam Wiggins, joined today by Jeffrey Litt. Hey. And Max Schoening, great to be here. Now, the two of you are working together with some others on an ink and Switch project we’re gonna talk about today, but first, I understand that there’s some cooking adventures going on in the lit household.

00:01:02 - Speaker 3: Yeah, so I’ve been trying to make my own stock lately. I’ve been reading this incredible book that someone recommended to me on Twitter called An Everlasting Meal by Tamar Adler, and it’s all about how to use leftovers and just like random stuff in your fridge to cook both as a way of not wasting, but also just cause it feels good. So I’ve been, you know, throwing random like carrot tops and stuff into a pot, and it feels really fun. That’s my recent cooking adventure.

00:01:29 - Speaker 1: Inspired by, we’ll get into it a little bit later with the project too, but partially by Jeffrey’s cooking. I’ve also started taking cooking maybe a little bit more seriously than before. Like I think one of the things that sort of distinguishes the amateur from someone who’s more seriously involved in something is consistency and my cooking was never all that consistent cause the loop of how frequently you repeat a dish when you’re just cooking sort of for fun is very long, right? So the learning is slow, so I’ve been getting into sous vide cooking. And just eating way more steak slash anything you can sous vide that I would like, but at least the results are getting better.

00:02:11 - Speaker 3: I’m a sous vide fan as well. It’s a major cheat code, I find. Everything is perfect every time.

00:02:17 - Speaker 1: I wish that had been my experience too.

00:02:21 - Speaker 2: Yeah, consistency and repetition, yeah, short feedback loops.

I was inspired by a book, I think it’s called the Food Lab, where basically the author does some, call it like.

Amateur science in the sense of taking common cooking claims, like should you salt meat before cooking it, or is it better if you don’t flip it, or you only flip it once versus twice or something, it would essentially just cook several side by side, varying this one thing. And then do a little informal taste test with his, you know, housemates or whatever, and sort of like try to answer that question, and many times found out that, or at least had the finding, let’s call it, that things that people swore by didn’t really actually make a huge difference in the outcome, but that idea for myself, I think even our Mutual friend and colleague Peter Van Hardenberg introduced a version of that in the Hiroku offices when he would do a little coffee workshop and essentially like brew a cup of coffee with several different approaches, you know, here’s the Chemex, here’s the French press, here’s the, and then you could taste them side by side and have new appreciation for the way these different techniques change the taste of the same source bean.

00:03:34 - Speaker 3: Yeah, I love that mindset. I think what I see as the challenge at home cooking is, you know, to bring in some of that idea of getting better and being a little rigorous without making the whole thing too overcomplicated and kind of perfectionist. There’s some aspect of amateurism and just having fun with it, that’s sort of the whole point to begin with. So I think that’s a fun balance to strike.

00:03:57 - Speaker 2: So longtime listeners of Meta Muse will know that we’re shaking up the format a little bit here.

This is our first time with two guests. It’s usually me and Mark as co-hosts along with one guest, and this is partially my theory that it’s a little hard for listeners to adapt to two new voices, but in fact, you two are not new to our guests potentially.

Max, you are one of our very first guests all the way back in episode 8 when we talked about principal products.

And Jeffrey, you joined us for somewhere around episode 34, where we talked about bring your own client. So, anyways, I thought it would be fun, especially because both of you work together on this research project to get you here together. So you can go back and listen to those episodes if you want the full backstory, but maybe you could each give a 32nd summary bio of yourself before we dive into the project itself.

00:04:51 - Speaker 3: Yeah, so I’m a grad student at MIT as well as a collaborator with the In and Switch Research lab, and my research mission is to figure out how to make software more customizable so people can edit the tools that they use and make their own software, and the project we’ll be talking about today has a lot of resonance with that theme, so excited to be here again.

00:05:12 - Speaker 1: And this is my first foray into research. I’m a software designer. I’ve worked on things like Hiroku, GitHub, cloud app, way in the past, and generally I like to summarize. My efforts as I like to make things for people who make where make is the developer build tool.

00:05:33 - Speaker 2: And our topic today will be dynamic documents, which indeed is what the potluck project that you both worked on and recently published about is all about. So we’re hoping to dive into the specifics of that project as well as some of the research process behind it. Maybe we could start out with a description, kind of the, I don’t know if elevator pitch is the right way to talk about a a research project, but a short summary.

00:05:58 - Speaker 3: We’re not raising funding, but I can give a summary, sure. So, Potluck is a substrate that we’ve been developing, where the goal is to turn regular old text documents into interactive tools that help you in your life.

And so there’s this idea that you can just start by, you know, jotting a note on your phone like you might in an app like Apple Notes, and then gradually you start enriching that note with little bits of computation and interaction.

And if you keep doing that for a while, you might end up with something that looks suspiciously like An app that you might download from the app store, but it’s not like someone else made it for you, it’s sort of organically evolved out of just a note that you started writing, and, you know, some examples of the kinds of things that we’ve thought about in the substrate are You’re writing down a recipe that your mom told you for how she makes her dumplings, and then you decide, oh, I’m gonna have a party, so I want to make 5x the recipe. What’s like 730 g times 5. That’s something that a computer should be able to help you with, right? But if your data is in a text note, how do you bring in the computer to play its role and help you out a little bit? We’ve developed these primitives where you can start injecting these little bits of computation as you need them into your text note. And so, that’s kind of the overall idea of the project.

One analogy that I think is helpful to understand the general ethos of it is spreadsheets. I’m a huge, huge fan of spreadsheets. I think they’re a really empowering medium that people interact with pretty typically on a computer these days. And the cool thing about a spreadsheet, right, is that when it starts out, it’s just a bunch of numbers in a table. It’s just data sitting there, and it’s already useful in that state. And then gradually you might add a little formula, you might add a V lookup, and if you keep doing that, by the end, you might end up with this ridiculously complicated app that’s running your whole business, but it didn’t start out that way. It wasn’t planned to happen that way, it just started out as this little bit of data that you were storing, and it naturally evolved, right? So that’s kind of the general idea.

00:08:04 - Speaker 1: Yeah, there’s the well known meme of this insert startup could have been a spreadsheet, and I think in this case, you could probably make the same argument that this app could have been a note.

And in fact, I would love to be able to do this at scale, but like if you just open the average users notes apps, what kind of notes do they take and, you know, throughout the course of this research project.

That was sort of a grounding force of, oh, what kind of notes do people keep track of and so we started looking at, you know, like Jeffrey was saying recipes. At some point we did workout tracking and plant water tracking and like collecting your favorite hikes and so on, and they all have this sort of very innocuous beginning. You’re not planning to make something big, you’re just sort of planting a little seed as a note and What was frustrating for us is, at some point, if you then want a little bit more help from the computer, you usually have to move it out of the notes app, which is a little bit sad because it’s this big drop off, and so that’s kind of what we’ve been looking at, like, how do you make that go away.

00:09:14 - Speaker 2: And I love the diversity of use cases outlined in the essay.

You focus on this cooking use case as a sort of a central one, even baked into the name of the project, but indeed all of these different kind of personal tracking stuff that tends to get scribbled down in notes and and in particular notes in your phone, text notes in your phone, that they’re not very structured, you’re trying to capture them in the moment and move on.

And certainly many of those are things I have done, but also there is a whole industry is a way to put it, category of app which is trackers. So, yeah, hike trackers and run trackers and sleep trackers. And yeah, fitness trackers, step counters, weight trackers, you know, and sometimes that’s paired with, I don’t know why, you know, Fitbit has their Wi Fi connected scale, and when you step on it every morning, it automatically records the data, but then a weakness for someone who is both curious and has some light programming capabilities is actually getting that data out or doing something with it in a more flexible tool like a spreadsheet. is often pretty difficult. Actually, Fitbit, I think even famously had a little bit of pushback for the, you had to pay for the feature to kind of like download your data as a CSV and even then it feels like this very discontinuous, OK, I’m exporting now, the data, who knows what format it’s even in, and there certainly can’t be a continuous using of the app, inputting of the data, and then also I’m gonna put it through my own, call it personal analytics.

00:10:43 - Speaker 3: Yeah, and I think another weird thing about this ecosystem of trackers is that it sort of splits up your life into these very specific categories, right? So, for example, after my workout, what if I have a nutrition shake and I’m tracking my nutrition, I need to switch from my workout tracker to my nutrition tracker, and there’s this sort of world that each app considers its space that it doesn’t go outside of, you know? This also happens when, you know, We looked a lot at recipe apps because we were thinking about cooking as one of our main domains with this tool, and a lot of cooking apps start out very simple with tracking your recipes, but then there’s sort of a natural force to bloat them with extra stuff, so, You’ll add grocery list stuff, and you’ll add menu planning, and, you know, meal planning for the week, and all these.

00:11:29 - Speaker 2: All of your friends, what are they cooking?

00:11:31 - Speaker 3: Yeah, like social, you know, and I feel like we’ve all experienced this tool starts out nice and small and grows in weird ways.

And what I think is important here is, I’m fine with it growing in the ways that I want it to be useful.

What’s annoying to me is having 100 things crammed in there that I don’t need and can’t remove from the tool. And then on the flip side, the one extra thing that I do need, I can’t add myself, right? So, we sort of like, let the developer of each tracker decide what does cooking or what does workouts mean, like, what’s the scope of that activity, and it’s really hard to permeate that kind of boundary that gets set there, and so that’s one of the problems that we were thinking about in developing potluck.

00:12:12 - Speaker 2: So some of the key concepts here, dynamic documents is obviously a spreadsheet is a dynamic document, but the idea here is taking text, plain text, which is incredibly universal.

Everybody’s phone has some kind of plain text notes app just kind of built in by default, but then you can use gradual enhancement to add some computation and make it something dynamic while keeping that same basic medium of just simple text you can manipulate.

That you also talk a little bit of the essay about personal software, which I think is precisely this concept you’re just describing here, which is rather than my run tracker being an app that I download from the app store and I’m more or less just have to use it as intended by the developers, that I can use the computational medium to build a quote unquote application that just suits my needs, is truly personal.

00:13:05 - Speaker 1: Yeah, personal software is, well, first of all, I think it’s getting more mainstream in the sense that if you look at a lot of people’s notion, usage, and all the other insert, you know, personal knowledge management tool here where people are sort of aggregating all of this stuff in their life into a personal OS and I don’t know where the appetite comes from. I don’t know if it’s tied to increased computer literacy, at least some form of computer literacy, or it’s people have been burned by their favorite app changing either by adding too many features or just being deprecated, but there seems to be a lot of energy around it and so one of the things that is surprisingly Or rather, something that you wouldn’t think about right away is when you start building these apps from scratch from a note, you never really notice that you’re actually making a big complicated thing. You’re just starting out with some text and at some point you start adorning that text with some functionality and you just keep going and going and going, and at some point you wake up and you’re like, wait, this is actually quite complicated logic.

Am I a programmer? And for us, that sort of was quite important, right, like embracing this notion of personal software that is truly yours, not from some team somewhere in Silicon Valley or wherever else deciding what’s best for you.

And I think Jeffrey, you gave this analogy early on to like imagine our homes were Just furnished completely by other people, and all the objects in there just are sort of almost immutable, like we would not have that, and we do with software, and so I think nudging at that is super interesting.

00:14:53 - Speaker 3: This is one of my favorite ways to Open up my own mind to how weird software is, is to use analogies to other parts of the world, you know, I think we sort of have gotten so familiar with these metaphors of how software is organized.

Like, in some sense, in this potluck work, what we’re doing is arguing against the idea of applications, right? Which is a really weird argument to make to a typical computer user, you know, it’s fish and water, like, what do you mean? I love apps. Apps are how we do things on computers, but It doesn’t have to be that way.

I mean, Alan Kay, who’s responsible for a lot of the metaphors we use in personal computing, has, I think, said that apps were like the biggest mistake that was made in software ever, or something like that.

You know, another analogy to bring it back to the food thing, I think, is restaurant versus home cooking. And the reason I like that analogy is that I think it gets that, I’m not trying to argue that we should ban restaurants. I love going to restaurants. It’s more like, if you imagine a world where All you can eat is restaurant food every day for every meal, and you think about what kind of society that would be, it starts to feel a little weird, right? When you go to a restaurant, you are putting a lot of trust in someone else to give you a good experience. You’re accepting kind of a restriction in choice, whether that’s like a full on oakcase, you know, meal, or even, you know, picking from a menu with 10 items is very different from going to the grocery store, right? But also, you’re acknowledging maybe that chef can do things I can’t, and maybe I’m tired today and don’t want to cook, whatever the reason may be, it’s nice.

But it’s also a certain kind of limited experience, I think.

And when I look at home cooking, I see a totally different set of trade-offs and values almost, where I’m not trying to become a professional, I’m not trying to make the best thing, I’m just trying to make something nice for myself that I like, and, you know, for my family, whatever. It’s a very different scale and and feeling, and I think that’s sort of the right way to think about, you know.

There are always going to be tons of professionals making software, and I think that’s great. I love Apple products where someone in Cupertino has thought for a year about what the width of this button should be. I’m not against that, it’s just that I think there’s also this complementary role for a different way of thinking, especially in these more personal domains.

And one last thing I’ll say about the home cooking analogy that I think is interesting is that it’s a very cultural thing. If you imagine a world where everyone always eats at restaurants and you tell someone, you know, why don’t you start cooking in your house? They might be like, well, I don’t know, that seems really hard, like, all these chefs have spent like years in school or whatever, and, you know, you can see the analogy here to, like, currently software is so professional and difficult, that it just seems unthinkable that everyone would be making this stuff themselves every day, but I think we can imagine a culture where that’s a little different, and, you know, try to promote that kind of Thinking and culture more generally, and I think that would be a good thing for software.

00:17:51 - Speaker 2: Yeah, I think the restaurant versus home cooking comparison is a great one, and also just reflects the fact that in the scheme of things, computing is just so new, and we don’t necessarily know how it fits into our lives and our society and how to relate to it, and we’ve ended up in this, you can think of it as a local minima or just a particular circumstance of time, which is that software is built by these professionals who are typically far away and building for many, many users. That’s not where we started with computing, and I certainly don’t think that’s where we’ll end up, but hence the reason to invest in research to take us in this direction.

Now one thing I think your project touches on that’s an interest of mine, obviously I would lump this under end user programming, something we’re all interested in essentially bringing programmability of computers to a wider audience, not necessarily in a professional app building context, but just in the sense of embracing the dynamic medium. But I feel one of the big unsolved problems of end user programming is really just getting it into a context where people can use it. There’s many, many really amazing research projects and prototypes and etc. where if you go into there, I don’t know what, here, launch this small talk browser and once you’re within that world, everything is malleable and composable and you have total power, but it’s not connected to anything you do in your life. And one thing I like about how your team went about this project is that You’re starting from text notes, which are on your phone. Now, it’s sort of an unanswered question is how exactly this computational medium gets into the notes app or whatever, that maybe it’s not a part that’s figured out, but very hypothetically, going from, I’ve got this text file or a series of text notes, and I wanna layer this dynamic medium on top of it, feels like a lot less of a jump than many of the other kind of programming accessibility research that I’ve seen.

00:19:47 - Speaker 1: Especially cause if you actually look, for example, Apple Notes, right, it already has hints of these data detectors. If you type a phone number and I don’t know, a few other dozen types of content, it automatically finds them for you and underlines them, and then you can, you know, tap and initiate a call and potluck just takes that notion to an extreme by saying, well, first of all, I can write my own detectors cause I don’t just want to find. A phone number I might want to find the quantity of a recipe and then it also just doesn’t limit you just to the oh I can tap and initiate a preprogrammed action. I can do something else with it, a calculation, fetch some different data and so on, but it’s a very gradual enrichment of the original note and it’s also already somewhat at home on iOS. I don’t know, Jeffrey, if you want to talk a little bit about the data detectors and the origin.

00:20:47 - Speaker 3: Yeah, totally. One of the more interesting kind of related work references that we found while we were thinking about this stuff was, what I think is the original paper describing the seed that has become these, you know, phone number recognition and stuff in modern Mac OS and iOS.

But there was a research team at Apple in the late 90s, which included Bonnie Nardi, who’s sort of well known in the end user programming space. And it’s really interesting seeing the original rationale they had for how they got to this idea of data detectors.

Their starting point was thinking about, OK, how can computers help us do stuff and be intelligent helpers, and they Sort of draw this distinction between two styles of how the computer can help you.

One is the computer just does stuff for you, or like, you know, you sort of vaguely say what you want and it doesn’t, sort of more of an assistant metaphor, you can imagine, you might not even know it’s doing stuff, it’s just behind the scenes. And I think this sort of corresponds to some of the modern ways that people think about, oh yeah, AI will just do it for you type of thinking.

But they realized that actually, both at the time that was totally infeasible. Computers weren’t good enough, weren’t smart enough to actually pull that off in a satisfactory way. And they also realized maybe it’s not quite what we want, and they went down another path, which is, let’s just have the computer find stuff for us that we care about, like, dig around in all the things on my desktop and find useful information, and then let me decide what to do with it.

And You know, as Max was saying, I think their view of what you can do with the information was relatively like straightforward. You just right click on a phone number and you hit call this number, simple interactions like that. But still, this idea that the user was in control of what to do with the information. And so, I think that’s a really nice kind of design goal for these sorts of systems is carefully balancing what are the parts that we want to be automated versus where the moments that we want to be in control, you know.

00:22:49 - Speaker 2: And data detectors is the term from, I think that paper in the potluck essay, you call it extensible searches, is this a rebrand to be a little more familiar to current audiences, or do you see that as actually, it’s because it goes beyond these more automated kind of default types, like a phone number and address?

00:23:10 - Speaker 1: With a lot of the stuff, it’s very serendipitous about how it happens during a project, and we initially didn’t even start out with potluck having these continuously running searches. It was much more of a manual process.

In fact, I think to this day, if you look at the code, it’s still like cold highlighters because we started out with this notion of, well, you have a note and then the sections that you care about, you’ll just highlight and you have different colors for highlighters to start imbuing those highlights with computation.

And someone on the project at some point sort of said, oh, well, why don’t we just run a search against it? And at the time, I think we didn’t even call it search, it was just a pattern. But if you think about how mere mortals would maybe think about this as well, I have like a Google doc open. How do I target a specific word in the Google Doc? Well I hit command F and I try and find it, right? And so that’s where this notion of search comes from, which is sort of the Most maybe human way of thinking about these detectors.

00:24:11 - Speaker 3: Another small thing I’ll add is that one of the really cool parts of the original data detector’s vision that we share, but it’s kind of been lost in the modern Mac OS version, is this extensible part of extensible searches is also really important, the idea that you can define your own.

You know, in potluck, this means that you can decide that these are the types of ingredients that I want to find in my document, and nothing else. I control the dictionary of what I consider foods, or I control the list of workouts. So if I write, you know, squat, my note will recognize that that is a kind of workout that I do, but all of it’s sort of very tailored to your life.

And the original data detectors paper had this too. They had this idea of, for example, you would teach the system, here are all the names of the conference rooms in my office. So whenever I write the name of a conference room anywhere in my OS, the computer will just know that that’s what that means. And of course that doesn’t apply to every Mac OS user, it’s more of a personal data detector that’s tailored to exactly my context.

00:25:08 - Speaker 2: Maybe like adding a word to the dictionary so that it doesn’t show up as a spelling error because it’s some nickname for something in my life that wouldn’t make sense to add to a global dictionary.

00:25:20 - Speaker 3: Yeah, exactly.

00:25:22 - Speaker 1: It’s funny to think about how frequently we have data detectors in the software that we use, right? Like on GitHub, if you want to reference an issue, you do pound 247, and that’s a data detector, famously like Twitter hashtags and at mentions were all not built into the software. They were just ways in which people invented small little microsyntaxes very fluidly.

And there is no primitive in the operating system that is not super far down to actually do something fun with those, right? And so by making it super easy right in the context of the note to write a new data detector and super easy, we can get into that a little bit later, is obviously a spectrum.

Ours still involves way too much knowledge of programming to be super easy. But the power of inventing those small microsyntaxes is super addictive.

Like you just start coming up with small things that only you are familiar with, kind of like in a notebook, you would have some sort of notation, and it’s just flabbergasting to me that operating systems haven’t embraced that at a more sort of a cross app boundary level, right? Like, I think to this day there is a class in Some SDK from Apple NS data detectors where I believe developers of apps can write data detectors for you, but you as a user have no influence over them, which is fine, like I guess some developer could build an app that lets you write your own, but partially what gets left behind there is that the same data detector should run across many applications.

If I have that, you mentioned a meeting room. If I have a meeting room name, then you should highlight it in iMessage in mail app in notes and my 23 other apps that I’ve just downloaded from the App Store, and that’s sort of lacking. If you read the paper, I highly recommend it. I’m a huge fan of BonnRD. It’s very sad that we don’t have that in our computers today.

00:27:24 - Speaker 3: And I think this brings it back to what Adam was talking about earlier around integrating with the rest of your tools, right? Sometimes. I think a really important point to make about this research project that we’ve made is that currently, just to be able to move freely, we’ve built this thing as it looks like an app.

You open this thing called Potluck, it is, you know, a web app, there’s a text box and you can do all these fancy things with the text, but the final form of this that we envision being good is not a separate app, it’s deeply integrated with all your other tools.

You could imagine any app that has text, you should be able to pull this panel of searches over and just start pointing it at any text in your system.

And we actually built a little bit of that into our prototype where we do some things where we actually interopt with.txt files on your file system. So you have a little bit of being able to open these text files in the text editor of your choice, work with them, and then when you look at them back in potluck, you see the interaction appear on top, but I think that one of the interesting open questions that we haven’t quite figured out yet is, how does it really work for this to be embedded at more of an OS platform level? Like, where do these search things live? How do you share them with other people? How do you share them across apps? It’s just sort of an interesting design challenge to think about there.

00:28:43 - Speaker 2: I feel like there is a commonality across many research projects that I was involved in and I can switch in maybe in general in the research world, which is if you could do it just in an app, you would try that, but the whole thing that makes a research is really this is something that should probably be operating system level or just cuts across.

This tech stacks or the tools you use or the devices you use in a way that isn’t really well supported by the current ways that things are divvied up or the way that we kind of compartmentalize the various elements of our computers and hence the only way to try them and see if they are plausible or good ideas in. or how they feel to use is to do them in this research context where you kind of have to hand wave and say, well, imagine this was built in your text editor or cut across all your apps or you know was there in the browser or you have a good way to share these things.

Would we want that? Would we like it? Would that help us? And that doesn’t get you all the way to what it would look like in the real world, but it certainly is a fair sight further than just sketching it out on a whiteboard.

00:29:49 - Speaker 3: The first one is figuring out what we want, prototyping the experience with enough fidelity that we actually have some idea of what platform primitives we would like to have available. And then the second part, which I think is at least as hard, is how would you actually enact that kind of change in the world. If the thing you’re doing is not trying to add another app to the app store, but totally change the structure of the app store, and all the economic incentives and the technical interfaces between things, that’s a very different shape of challenge, and so, yeah, it’s a lot.

00:30:24 - Speaker 2: Now obviously I’ll link the essay in the show notes, and there’s also a live web demo that’s pretty workable, I think, or at least in my experiments with it got pretty far, which is saying a lot for a research prototype, which tend to be, you know, focusing on the learning rather than the polished product. So I’ll link both of those in the show notes and people should certainly check them out. I’d be curious to hear briefly. On the findings and what you learned from building this and trying to use it in practice. Was there anything that stands out as surprising or unexpected?

00:30:58 - Speaker 1: I think there was this distinct moment in time where the prototype was actually good enough, that inventing your own syntax for something was very trivial.

You could just say something like find every line that starts with plant emoji, and then suddenly do something with it.

And I still remember it having the feeling of why doesn’t all software work this way. And so to me it wasn’t super obvious that personal microsyntaxes should be a thing, and the idea that the same way you can scribble personalized notations into a paper notebook, that we could bring that into software.

If you make it easy, right? If you don’t make someone go into some settings screen that’s 4 pages deep to say I’m going to change how this works, right, like usual programming, but just ad hoc, you’re like, oh, I’m just gonna start these next lines with, and, you know, famously look at markdown, like, I’m gonna start the list with asterisks. Well, I can just invent that. That to me was actually a very surprising finding is the ease of creation of a syntax and then the utility.

00:32:07 - Speaker 3: I think for me, one of the surprising things was just how nice it is to work in text. This might be sort of a bit of my programmer brain, you know, speaking, but I’m not typically one of these, you know, everything I do is in plain text kind of people, but I found that We’re so used to editing text. We have really strong muscle memory around, for example, things like, I can select some text, cut it and paste it somewhere else, or I can even paste it into another app, or I can undo, and I understand how Undo is gonna work. And all these little affordances are really mature in the systems we use, and they’re mature in our heads. They’re really strong conventions, and I think One thing we found is that when you build software on top of that really solid foundation that we all have, a lot of things just sort of fall out of that.

So, for example, when all of the state of your application and all of its UI live in a text file, you can just snip parts, move them around. If you undo your app has undo for free because it’s state is stored in the text. You get all these things out of that. And I think there’s some lesson there it feels about, I guess it’s about using the same well developed tool for many different things.

I’ve used the analogy before of, it’s like a chef’s knife, where it’s like, good at all these different things and someone put a lot of effort into making it really good and versatile. It feels like there’s something similar there going on with text, and It’s not a new insight. I think there’s lots of people out there who do Emacs or there’s all kinds of to do list apps, or, you know, budgeting apps based around text files. I think that’s a thing that some people have been experimenting with for decades, but it was surprising to us just how far you can push that into so many different domains. Of course, you can’t do everything with text. There’s lots of apps that would be ridiculous to even try making them potluck, you know, YouTube is not the target, but I think that’s fine, you know, there’s some Kernel of personal use cases that fits really well with this medium, I think.

00:34:05 - Speaker 1: Yeah, plain text, or just maybe text in general, is a surprisingly good layout engine in the sense that if you want to make personal software very frequently you’re gonna have to go and come up with your own layout.

Oh, I’d rather actually have these things at the top and not at the bottom of the screen. And I think maybe because as professional sort of software developers, maybe we go towards building a GUI with specialized inputs and forms and controls and all that stuff too soon because it’s just so much easier to explain to the computer what the user means if they use a specialized input tool like a button check box and so on.

But if that weren’t the case, if it’s just easier for the computer to understand what you mean as you’re typing in your note. Then suddenly text input is the primary thing.

And if you think about what we do on computers all day, including people who are not sort of in the industry, yes, Emacs and so on, great, but you spend most of your time writing texts to people, right? So if you can’t type on your device, then you can barely use it, which means most people who use devices spend a lot of type typing.

And I think we should encourage software designers and developers to lean into text way more than we do, and like you even see that possibly in this resurgence of the command K command lines that every app now implements, right? Like command palettes, which are also just text-based entry.

And so I think potluck maybe takes this to the extreme of saying, look, just write whatever you want, and then we’ll just teach the computer with you how to interpret what you wrote, and then you can do awesome things with it, and that’s kind of exciting to me.

00:35:47 - Speaker 3: In some ways, it’s like even one more step towards messy than spreadsheets.

Someone at the lab was computing a spreadsheet, I think, to sum up how heavy things would be in like a backpack for a hike, and At some point they realized, oh, I should just do this in potluck, because even the effort to put it in a spreadsheet table was just felt like a little bit of ceremony, like, spreadsheets are sort of clunky to edit on your phone, for example, whereas text, it just kind of, it’s one dimensional, so it resizes onto your phone, you just type characters in, it’s very low ceremony, and so if you can get the interaction you want out of such a messy data substrate, in some ways I think it’s like a good go to before you start adding too much structure.

00:36:29 - Speaker 1: Yeah, there is this, I think we link to it in the essay as well, a paper, deferred formalism, which sort of encourages you to not get into structure really early on, right? and text is great, like I can just put the cursor in the middle of a line, hit return and now I have two lines.

And if you think about some of the tools that are extremely popular, notion and so on, they always make that distinction of are you inputting pros and making a list, or do you want more structure to do some computation, which is, oh great, now you have to think like a DBA and the notion of being able to move between those two modes fluently, I think is really cool.

And at the same time, if you can afford to push the formalism as far back into the process as possible, right? Like, hopefully without the app hopping, right, of like, oh, I started thinking in Muse, and suddenly I want some more structure. Therefore we have to go get out a spreadsheet and at some point you’re like 6 level deep writing a rails app with a SQL light database and you just don’t, you know, it’s it’s not the way to go.

00:37:34 - Speaker 2: And now when it comes to structure versus free form, I do think there’s a feedback loop when you talk about microformats where you kind of are inventing your own little structure as you go, just naturally, even like writing in a notebook can be something like this.

Yeah, some of the trackers you mentioned there, one use case that came to mind for me was in the early days of parenthood, we basically had a log for things like feedings and sleeping and diaper. because it’s very useful, especially with handoff between caregivers to just at a glance, be able to look at this and see when was the last time they ate, when was the last time they slept, because that tells you a lot about trying to figure out whether they’re crying right now, what need they’re expressing when they’re crying right in this moment, as well as other maybe slightly longer term analysis in terms of like, OK, are we getting enough sleep each day, for example.

But there does tend to be a feedback loop if you do add the rigor of the computer trying to parse it, even if I’ve written that search for myself, then that is going to enforce as a strong word for it, but encourage me to use a format that can be easily parsed and to be consistent with that because I make my job on the called the programmer side or the adding the dynamic aspect to it a little bit easier. Did you see something like that in your user testing?

00:38:55 - Speaker 3: Yeah, I think there’s a really interesting tension here where it’s exactly what you’re pointing out, where on the one hand, we don’t want it to feel like programming.

So in textual programming, especially for beginners, you’re typing in characters, and there’s a very, very strict set of rules defining what’s valid and invalid, right? And if it’s valid, you get nice syntax highlighting and everything, it’s great, and if you have one comma missing, everything falls apart, and so we I thought that it was really important that you don’t start feeling that way in your text notes.

It should generally have the sense that you can just type the way you normally would. But of course, on the other hand, we still need to figure out what you meant. And so you need patterns that are accommodating enough to let you write, but also rich enough to figure out what you meant and extract the meaning. I do think one thing we realized is that there’s a big difference between applying patterns later on to some text that’s already sitting there, versus having them being applied live as you type. Because in potluck, as you type, for example, if I type 5 minutes into a note, by default, potluck has this time recognizer built in, which will search for all the durations you add to a note. And when I type 5 minutes, this underline just appears and clicks into place, and I sort of get this live feedback that I’ve typed a duration that the system has understood. And we just found that that felt really good. It feels good to have the system give you that signal that it recognized what you did. And obviously, if you were expecting it to recognize something and it didn’t, then you realize that because of the lack of feedback. And so, what we found in our experience using this thing was that if you have the searches running as you’re inputting the data, it does have this natural nudging force of making you aware of the structure a little bit and maybe being a little more mindful of where you put. New lines or things like that, but again, shouldn’t be too rigid ideally.

00:40:45 - Speaker 1: Adam, you do bring up an interesting point. I think partially why notion is so popular and such a great tool is that it does invoke a little bit like this collector mindset of I’m just going to collect, you know, whatever you’re into and make a nice table so that I can actually reason about it as a collection instead of individual items, right? And like I always joke that sort of computers are really, really good at doing stupid math and for loops. And maybe one way of thinking about this is if you look at the user interface for potluck, on the left hand side you have sort of this messy, I’m just going to type stuff out, and as you write searches that match against the document, it populates a table and that table can have arbitrary metadata, right? So I can add a new column and say actually for this timer, I’m going to add a different property and The idea of having both, both this sort of reasoning about things in collections, large, you know, all ingredients or whatever, and the idea that I don’t have to do that from the beginning, or if I change my mind, it’s not such a big deal, is really appealing to me because I think we usually switch modes from reasoning about the individual thing to the collective thing and back and forth and back and forth and software today just makes you. Sort of jump through hoops if you’re switching between one or the other, and potluck tries to, as best as possible, sort of make that fluid.

00:42:12 - Speaker 2: One thing I wanted to ask about is, in the future work section, you talk about machine learning and language models, which is a pretty hot topic among certainly the tech world broadly and also in the tools for thought space. Since you are focused so much on text as well as detection, what role do you see that as having either now or in the future?

00:42:34 - Speaker 1: It’s really interesting timing because as we were writing the paper and like doing the research project, all these big language models and like stable diffusion and a bunch of other things sort of came out and became sort of accessible to like hackers, I would say. I mean they have been for a while.

But at some point we were thinking about, well, we have these searches and the way we’ve implemented searches both sort of from a time perspective and maybe a little bit of a philosophical thing that we can get into, are all, I don’t know, like rejects, we have our own pattern language and so on, and you can if you want to write rejects. It’s obviously not super approachable to mere mortals.

So how wouldn’t it be cool if I could just have a note and say, find me all things that are quantities of food. And then GPT 3 goes off and comes back and says, here are the ones that we found.

And I think it is so obvious that the data detectors will get so much better the better machine learning gets, right? And you can kind of get a glimpse of that future in the photos app on, I forget the current photos app on on iOS, where if you take a picture of a recipe index card and it says 24 g of sugar. It’ll actually, you can tap and hold and it’ll do a unit conversion for you. Now, it won’t let you do anything else because somebody decided for you what you should do with those 24 g. That’s the part where we would hope some other, like some maybe some more extensibility, but that’s just machine learning, finding the 24 g for you and you don’t have to do it, right? And so I think it seems somewhat obvious to us that all the detectors that are currently patterns will just become much more human friendly ways to describe patterns.

00:44:16 - Speaker 3: I will add one interesting tension that we were thinking about a bit. We didn’t end up implementing AI based stuff in the project since we didn’t have time to get into it, but we thought about, you know, do you want the AI to find the stuff for you, or do you want the AI to essentially write a reject for you? And those actually end up being pretty different things because predictability and speed actually end up mattering a lot.

When I’m typing, I wanna be able to learn.

You know, if I type this string, is the computer always gonna see that as a food or not? And if you have machine learning in the loop for actually doing the detection, It’s probably pretty hard to get guarantees around, oh, you know, it depends on where it is in the sentence, or how the model’s feeling that day, whereas if you have a more deterministic pattern, that gives you something that you could learn as a human, like how it works, and sort of learn to wield predictably, but there is a tough tension there because the predictable thing probably is gonna miss a bunch of cases that the ML could have found.

So, I think there’s an interesting design challenge there and how do you Get a system that does both of those things well.

00:45:22 - Speaker 2: And maybe an example of that from kind of an earlier phase of technology is autocorrect, which on one hand was this huge enabler to be able to type full sentences on a phone.

On the other hand, is the source of huge running jokes, you know, it’s basically the butt of jokes, which is like autocorrect, does hilarious things all the time, people are used to that, it’s part of modern life in a way that, oh, I pressed the wrong key on my keyboard. I guess that happens sometimes, but it’s so infrequent for someone who’s a reasonably competent typist, that it’s just not a point of discussion, and, you know, it’s one thing to use autocorrect to bang out a quick text message to someone, but if you’re a book author and you can sit down and write your book, Autocorcrack is not the right solution for you.

You’re gonna become a touch typist with a precise keyboard, maybe you get a mechanical keyboard with big chunky keys, because, yeah, you want precision from your tool and you’re willing to invest in that.

00:46:18 - Speaker 1: Yeah, I think it may have been Paul Shan on the team that came up with the funny analogy of, we were arguing about AI cause I think sort of just what role should it play in potluck and so on, and One of the things he referred to current AI models to is like, look, this is like the toddler stage, and if you go and say, hey, go toddler, find me the ingredients on this table, you’re probably not going to just blindly take them and then cook a meal. But at the same time, if you can send off 10,000 toddlers to try and find the ingredients on the table, and then you check their work, seems pretty reasonable, right? And so I think that the idea of having the ML try and suggest something to you. But then you check the work, commit that and say this is the correct thing that you found, then it’s a totally reasonable approach, right? Like, I think GitHub co-pilot does this for programming, like you’re not writing a method call that at run time. Goes to GPT 3 and says please sort this list. It gives you the text to autocomplete that then you commit and run. I say this as now there are examples where GPT 3 calls itself to do stuff, which is both super exciting, but at the same time, you probably wouldn’t want that to be part of your stack all the time cause you can’t rely on, you know, the model upstream changing and suddenly saying that the car is an ingredient and yeah, but I think that tension. is good. I think we haven’t really figured out what the user interfaces for AI and for that interaction looks like, right? Like right now, all these interfaces are just slot machines. Like stable diffusion is just, it’s addictive because it’s a slot machine. You type in a prompt, you have to wait 30 seconds and then you get the variable reward of nice picture or not nice picture. But for a tool, I think you would want something a much more fluid and fast, right? You can’t wait for 30 seconds and you probably also want something much more predictable. I think it’s a future research project waiting to happen to say, in an environment like potluck, what role does AI play and how would you go about designing that?

00:48:23 - Speaker 2: Well, it’s all super fascinating stuff. I highly recommend reading the essay, trying the project, but now I want to switch gears a little bit and ask you both about the process.

What does it look like to, I guess, come up with research to work on in the first place and certainly within the can switch container, recruit the team and run the project and how long does it last and who’s on it.

And I’m especially interested to get both of your takes cause Jeffrey, you’ve done a bunch of Ink & Switch projects at this point, as well as been in the research world for a while, and Max, this was sort of your first exposure. You’re very accomplished in the commercial world, but this was your first exposure to both the research world and I and Switch. So, yeah, give me the rundown. What does the inside of this box look like?

00:49:08 - Speaker 3: Yeah, totally. I guess I can speak to how the project originated as the person who kind of started bringing things together here.

So I can Switch typically these days runs projects that are You know, around 10 to 12 weeks, which is pretty short from a research perspective, and what we try to do is bring together some small team of people for an intense period of time there and just really focus, you know, ideally everyone’s full time and just intensively work on some aspect of a bigger problem. And so, before that, there’s this phase that we call pre-infusion, which I think Peter talked about when he was on Metamuse, where there has to be some prep work to figure out, you know, who do we need on this project, what’s the question we’re asking. In this case, we knew we wanted to do something related to the themes of malleable software, you know, this personal tool stuff, but we spent a while kind of searching for, I think it’s important to have kind of a nucleation point of some kind, something you can latch on to as a place to start, especially, I’m a fan of having some concrete examples or use cases in the mix at that stage, because what I found is that if you start with your prompt being like, how can we reinvent the way people do work or something, it’s easy to get lost in the woods, basically, whereas if you can at least focus on one thing to start and then branch out from there, it makes it easier. And so, I think Peter, the lab director and I were having a conversation, and we’re both big fans of cooking, and so we just started talking about, you know, isn’t it kind of weird that recipe apps are simultaneously so popular and yet seem to do so little and have all these frustrating restrictions, and so we thought it’d be fun to run a project where the original prompt was kind of, could you make a recipe app yourself, and, you know, go from there. And a lot of the ideas that ended up emerging in pot, like, we didn’t really Set out specifically to answer that, you know, specific question, it just kind of more emerged from the original prompt. So that’s kind of on the idea side, and then on the people side, you know, I am a big fan of small teams where everyone can kind of do everything a little bit, generalists, especially in this case. I think one of the tough things about this kind of work is that there’s a lot of context you have to build up, and so I felt that it was important to get a team together that had at least been thinking about these general kinds of problems before. You know, if you bring a typical engineer onto a project and say, let’s get rid of apps, you know, that’s sort of a strange place to start, right? And so, anyway, that led to, obviously Max, as sort of a design focused person, and then Two other people, Paul Shen and Paul Sonnetta, who are both, you know, more engineering focused, but, you know, all four of us had previously thought about these themes, and so it was really fun to get this group together and kind of jam on, you know, each having a different perspective on what it means to make personal tools, but kind of find a way to blend them in a way that made sense. So, that’s kind of the general overview, I guess.

00:52:12 - Speaker 1: I think you had a comment early on when we were doing intros, because most people we hadn’t worked together yet, and I think you made the comment of, oh, it’s well, if any of the people on the team really wanted to, they could just make the whole thing themselves.

And I would have actually loved to see a parallel universe where we all separately would have tried to make a malleable recipe app, cause I’m sure it would have been very, very different than what we ended up coming up with.

But this idea that you don’t have to spend any time explaining basics and can just go into building right away is really important when you only have 10 weeks or so. And I mean, I loved working on this team, maybe my favorite team working experience I’ve ever had.

00:53:00 - Speaker 2: Wow. Now, how did you perceive Max, this kind of research angle where the end goal is not to ship something to end users, yeah, you want something usable, and even there’s a demo on the web, you can go try, but the goal here is not to build a product and iterate on that and bring it to market. How did that change the experience of building something for you?

00:53:22 - Speaker 1: It was both very refreshing and at times frustrating, so it was a little bit of a palette cleanser.

Most of the time when I’m looking at, you know, building software, it’s like, OK, when do you get to product market fit and what’s the economic viability? How many users, how are you going to make money, whatever, right? And this is not the case with research projects. There, I think the goals are much more, can you find a novel take that maybe explicitly wouldn’t work.

In the app store right away, because, well, either the tech’s not there yet, or you need to commit access to Mac OS and iOS to actually fix this thing, or Linux or whatever.

And so I didn’t really have any notion of what it was gonna be like. The only thing I knew is that all the Ink & Switch essays are badass, and surely something about the way these projects are run contributes to it. And I think it’s that weird tension between both, well, we’re gonna think big and do something that might not be viable right away, and at the same time we’re gonna ground it in that use case of, in this case, Jeffrey’s idea of a, well, let’s just make a recipe app. That’s our use case. How would you make that malleable instead of much more generic and, you know, inventing something that maybe no one will ever want to use.

00:54:47 - Speaker 3: Another part of the research first product thing that I find important is the end goal of this project is kind of idea transmission, like, we succeed if we change the way people think. And so, the way you explain the thing and frame it ends up being super important, which I guess that’s also true of marketing a product or whatever, but I think it’s just When that’s the main artifact that you’re going for at the end, it puts a lot of pressure on that angle of things. So, one process that I think we all agreed was really helpful is typically on lab projects, every 2 weeks, there’s a demo day, basically where you just demo what you’ve been working on to other people in the lab, and I think it’s really important to take those opportunities to sort of practice the story and try to explain what the heck are we doing, what problem are we thinking about, what’s our prototype right now. And just rehearse that every 2 weeks. And if you can’t convince other people at the lab that this thing makes sense or is good, you’re never gonna succeed at convincing anyone else, right? This is like the most high context, sympathetic audience you could find. And we did have a couple demos where people were like, what are you doing? This doesn’t really make sense, we don’t get it. And that was really, really helpful for kind of refining both the way we explain what we’re doing, but also, you know, obviously the work itself and sort of guiding the direction of it.

And I think that’s an interesting process question is like, what cadence do you work on? In some ways, 2 week cycles may seem pretty fast. A lot of researchers work on much slower sort of base cadences, but I find that I really like having an intense kind of pretty fast rhythm when you’re in this execution, or kind of intense momentum mode, and then Once you’ve finished this 12 week period, you can spend some time to like, walk around and think about what you’ve done, and think about what you wanna do next, and, you know, have a sort of on-off approach.

00:56:44 - Speaker 1: That tension or the 10 weeks, Jeffrey and I have definitely had some conversations about, is that too short? Is it too long for a research project and sort of, I think my view initially was. It’s like, 0, 10 weeks is too short to do any meaningful research, and I think that’s still true, except that you shouldn’t consider those 10 to 12 weeks as the entire research.

It’s a season in an 11 season lost sort of show, right? And there’s this thread across all I can switch projects and Potluck will infuse other projects going forward, and I think if you bring that mindset, then suddenly the 10 weeks are really great because it’s this forcing function of just not wasting 2 years trying to see if there’s a there there. You have 10 weeks, go ship something, publish it, and have it torn to pieces because it’s not good enough, right? And if you’re not embarrassed, you’re shipping too late. I am definitely embarrassed by some of the UI and the UX and the Maybe complexity that exists and it’s not a product, but the idea that it kickstars, you know, a couple of other seasons of development, I think is a good framing, and in that case, the 10 weeks, the intensity, daily stand ups. I was only on it halftime, everybody else, which I do not recommend, everybody else was on it sort of full time, and the intensity is truly what leads to this pressure cooker environment of like building something that’s both good enough that you want to play with it. But not a thing that is ready for any kind of adoption by people outside of the lab environment.

00:58:24 - Speaker 2: I’m curious about the transition from that, yeah, 10 to 12 week more intense building phase to, OK, now let’s take what we’ve learned and turn that into a written artifact or it could be sometimes a talk, but in this case it was an essay. I guess some of the question is, is all the team involved with the essay or just the writing? How do you know when you actually have something good to write about? You’ve learned something useful, which maybe could happen halfway through that 10 or 12 weeks, or maybe you get to the end and actually don’t feel like you have a lot to say. How does that whole transition work?

00:59:00 - Speaker 3: I love the writing phase because it’s where you get to figure out what you’ve done, and I think it’s really funny. This is such a cliche, but like, you start writing and it’s like, wait, what do I want to say? And it can get really confusing, and I think in some sense, even once you’ve done all the work, you haven’t actually done the work yet of figuring out what you’ve learned from it.

And so, on this project, what we did is we tried as we were going to Prototype the paper, kind of. We, you know, recorded little talks explaining the project or like, wrote notes of, like, here’s how I would explain it today.

But even then, when we got to, you know, the end of this intense period, there was still a lot of mess to work through, and we’ve all been involved in co-writing the piece, and I think that’s sort of important to the extent that, again, the real value that this thing is trying to provide to the world is like, here is what we learned, and the writing process is where that gets clarified, you know.

00:59:53 - Speaker 2: Do you find you wanna go back and make changes to the software as a result of things you’re writing, or especially screenshots or videos you’re including?

01:00:02 - Speaker 1: Yeah, when I read the essay now, I’m like, OK, obviously an exaggeration, but this is all wrong. We have to start again.

This is how we would design it, and it’s really not that it’s all wrong. It’s just that you want to do so much more to it, right? Like now is the time where you would simplify and so on, and the writing is the clarifying piece for me, the seeing what findings we actually think we found. Even if you set the essay aside for a couple of weeks and then look at it again, it’s like, do I still agree with this? Is this still a sort of meat and a finding that’s worth publishing, or was it incidental? But yeah, for the most part I have the feeling of great, this is potluck one. Let’s get both a sledgehammer and like a scalpel and figure out which parts need to be reworked. Less so in the essay itself, more so in the experience, so that the findings just shine through the experience very clearly, which is obviously a form of iteration, right?

01:01:03 - Speaker 2: I think this projects that end, you cut it off and you have the space and time to not only think about it, but then try to write it down, indeed may spend as much or even more time in terms of calendar time writing as you did building. is something that is unique to research, yeah, when you’re building in the commercial world, whether it’s a small business or a startup, either way you have the kind of time is money pressure, you gotta keep shipping, fix those bugs, satisfy those customers with the features they’re asking for, which is well and good and as it should be, but often it doesn’t give you that time for reflection.

And by drawing that really hard line, you have the chance to learn, and then in that next iteration, it can be not just the scalpel, but the sledgehammer something very different, even discontinuously different in a way that maybe it wouldn’t be with a product.

01:01:57 - Speaker 1: Yeah, I think you could maybe fake this for a product as well, in the sense that everybody or a lot of people are familiar with the one pager idea of look, we’re gonna write the one pager of what this product’s gonna be, and you include all the hopes and dreams and all this stuff that you may or may not do, and then you actually go work on the thing, and after like 3 or 4 weeks, you’re like, OK, this has diverged from the one pager.

Go rewrite it or perhaps write the announcement blog post of the thing. And sure, you could write the announcement blog post of the thing from day one, but I actually think the refining and constantly evaluating, is that actually the thing that’s core to this idea, or is it something else? You could bring that same kind of mindset into a product development process, I think, especially for less, oh we’re making version 27 of something and more 0 to 1, maybe 0 to 2 kind of environments.

01:02:52 - Speaker 3: This is one of the reasons that I love reading good academic papers. Not all academic papers are good, but the good ones are really, really good, and I think it’s because, if you think about how much time has gone in per word in that document, you know, it’s very possible that a team of 20 people spent years working on something.

And then obsessed over how should we explain this, what order do we bring things in, one of the best examples, and it’s just like the opposite of a tweet, you know, it’s like, how much effort can you put into explaining something well, it’s really rewarding to read things like that when you can tell someone put in that much effort to the writing.

So, I also find it fulfilling on the other side of trying to aspire to write stuff like that.

01:03:36 - Speaker 1: To bring it back to cooking, and Jeffrey’s latest obsession, it’s like making stock, right? You just keep working at it until it’s reduced and all the flavor is in there, and I think that sort of writing process was incredible here for me, at least, cause it’s the first time that I’ve even just adjacently participated in writing a paper.

01:03:56 - Speaker 2: Thinking about what it would be like to take this project and turn it into an app that you could use that kind of fits in with your operating system.

Max, you made the point earlier that you would need to commit access to Mac OS, iOS, essentially the computing platforms that we use to bring it in the way that you’re really envisioning it.

And I think more broadly than that practicality of you want to modify the operating system, it’s really a rethinking of how the pieces of our computing world fit together. So again, it’s much easier to create a piece of software if it fits cleanly into a box. Here’s a website, here’s an app on the App Store, but here when it cuts across those layers or recombines them in a new way, and that’s part of what makes it exciting, but also makes it very hard. To go from this, let’s say idea space slash hypothetical embodied in a prototype to something that’s in the real world. What are the implications? Where could we go if we were inspired and excited by this vision for computing?

01:04:55 - Speaker 1: I think the notion of unbundling the app is really interesting to me. This idea that you have your home screen and we have these little squares, and those squares are quite literally fused together behavior plus data and getting data from one square to another is sort of this orifice of the share sheet is just not good enough, I think, and If you want to unbundle the app, then you have to sort of make the data substrate. In our case it was, you know, plain text, much more shareable between, let’s call them apps for a second, but at that point the apps are really just views with a specific emphasis on what you’re trying to accomplish, right? Like, is it more image focused, is it more text focused, is it video focused or whatever, and they all operate on the same data substrate.

Therefore, each of the apps sort of lose importance and the unbundling is more about the interop between them. And I think it’s likely that we’re already seeing the beginnings of it.

Like I think if you do look at even though we don’t have commit bit to iOS and Mac OS and so on, you see this sort of like widgets and all these things that are just taking specific views for A set of data and putting it on your home screen or on your lock screen or whatever. And that’s the notion of just breaking it up, putting it on the device that’s most useful, maybe it’s your watch, maybe it’s your phone, maybe it’s your computer, and optimizing for that use case, but the data lives sort of in this bucket.

Now, if we just said, hey, let’s break that up, and the data doesn’t live in the app bucket anymore, it lives on this magic data substrate.

Suddenly you get an experience that is much more like we are used to with working with plain text and notes, right? Like if you told a developer, you have to use this editor that we’ve blessed, you’d be like, no, I want to use a different app and you can because it’s just plain text, right? And you can use anything from them to X code to Visual Studio code. The same can be done for other types of applications if that shared substrate is sort of more powerful than maybe just dumb files on a disk.

01:07:10 - Speaker 2: Who or how do we create this one magic data layer to rule them all.

01:07:16 - Speaker 3: If any of us knew the answer to that, we’d be billionaires already. Well, so one possibility around that that I find exciting is, as you know, Adam, I can Switch has done a lot of work around local first software, which is a research theme that I see very deeply entwined with that thing Max was just talking about.

The local first cross tool file system, I think, is a thing that needs to exist.

And so, I’m kind of curious to see how That research that the lab has been doing for a while, starts to entangle with these ideas from potluck, as well as even, you know, the lab’s also been doing work for a while on thinking with ink and tablets, right? And we’ve actually been noticing recently that some of the ideas from potluck seem to be related to that work as well.

There’s a shared theme of working in the way that feels natural to you in a very messy, sketchy way, but still being able to bring in computation when you want it. And so, I’m excited to, you know, see going forward how those different threads of work can kind of combine with each other, and maybe someday we’ll figure out how or who is going to build this better computing platform that we all want.

01:08:26 - Speaker 1: I’ll plug Jeffrey’s I can switch essay, Cambria, which talks about this notion of bidirectional lenses on data so that you have interop between two things that may interpret that data somewhat differently, and the surge of AI and ML actually gives me hope that we can Not necessarily at that level, but at the human level, make the interop between different pieces of data much more fluid, right? Like the idea that today you have to reason as a computer user that a PDF is fundamentally a different thing than a JPEG or a screenshot is very unreasonable if you think about it, right? That’s just, I think Jeffrey likes to call it programmer bullshit. I’m not sure if we’re allowed to swear on the podcast, but AI makes the input and the output much more lossy, right? It’s like, oh yeah, just interpret my screenshot. It’s text, why can’t I copy and paste it? And so I think that plus some advances in sort of the CRDT space and actually gives me hope that we could achieve something that is much more of a Less understanding what a signed integer is and more, hey, look, I have a number, go figure it out, computer, that’s what you’re there for.

So, I don’t know, the future seems bright.

01:09:46 - Speaker 2: Well, let’s wrap it there. Thanks everyone for listening. If you have feedback, write us on Twitter at museAppHQ or via email, hello at museapp.com. And Max Jeffrey, thank you for stretching our imaginations for what we can do with dynamic documents and text and end user programming and well, I’m looking forward to that next generation computing platform.

01:10:10 - Speaker 3: Thanks so much. This was fun.

01:10:12 - Speaker 1: Thanks for having us.