Transcript

00:00:00 - Speaker 1: That to me was the magic of the iPad, the direct manipulation of the iPad with my hands. It just felt so human in a way that the computers and even the phone never did.

00:00:19 - Speaker 2: Hello and welcome to Meta Muse. Muse is a tool for thought on iPad, but this podcast isn’t about Muse the product, it’s about Muse the company and the small team behind it.

I’m Adam Wiggins, joined today by my two colleagues, Mark McGranaghan. Hey. And Leonard Sversky. Hi. And I’m very excited to say that we have just booked our lodging and flights for our first in-person team summit in a year and a half, is it? The last time was Arizona in early 2020.

So we’ve been doing all our summits, which is a very important way that we plan our work and just bond as humans get out of the day to day a little bit. We’ve been doing it all virtually, but that just is not the same. So we’re gonna be meeting soon in France for a nice get together and chance to really think some big thoughts. Look forward to seeing you both and our two other colleagues in person.

00:01:17 - Speaker 1: Yeah, it should be awesome.

00:01:18 - Speaker 3: I’m kind of proud of us for actually making it all this way basically, but yeah, it feels like we really need and would really benefit from seeing each other again.

00:01:27 - Speaker 2: For a team that scaled a lot, we had the benefit that the four of us already knew each other in many cases very well, because we’ve been working together for years, we already have those human connections. It’s easier to translate that to the virtual space. But I think if you had a team that was adding a lot of people swiftly, yeah, seems like a challenge to scale the culture, to keep the creativity and vision, and all the things that just tend to come from being able to not just see each other as moving squares on your screen, but as real full three dimensional human beings.

So our topic today is the future of the iPad. So Muse is, at least at the moment, an iPad only app, so clearly we’ve bet our business on it, and we see big potential in the iPad as a creative tool, not just a consumption device, but something you can use to create, do work, be productive, and of course, for our purposes to think as a rumination space. But we’ve been at this a few years now, it’s interesting to look both at the history of how the iPad has evolved even as we’ve been on it.

Then furthermore, at this moment, iPad OS 15 is in beta. It’s got some enhancements to the multitasking capabilities, which is sort of a power user capability, and all that just, I think, had me at least as I was using the beta, reflecting on how has this platform evolved.

From our perspective as app developers as users that want to see it be a great creative tool.

So I guess the first question that a lot of folks tend to ask. And I think it was last year the iPad had its 10 year anniversary, and there was a lot of articles about what does it mean or where are we at or how has this platform evolved in this time, and I think the tenor there was generally negative. I’ll link a few, but Strateteri, for example, has one called the Tragic iPad, and they basically say it’s a device that never found its purpose or never found its real role. It’s sort of too big to be mobile and fit in your pocket the way a phone does, but it’s not as powerful as a laptop. This thing doesn’t have a clear role in people’s lives, at least that’s the way that was presented then. How do you both see the role and who it’s for question with the iPad.

00:03:35 - Speaker 3: I think the fact that it doesn’t have a clearer role is both the appeal of the iPad for many people that it can be a lot of things and a lot of different things to everyone, but it’s also, especially for us, the developers, it’s also the problem, right? That we don’t really know what Apple has in mind for the iPad, but it wants the iPad to be who it markets the iPad for. And so it’s hard to really think about the future of the iPad and be certain what kind of app you should build for it.

00:04:03 - Speaker 1: Yeah, to my mind, the verdict is very mixed here.

So I think the iPad has succeeded as this unique third form factor that’s somewhat mobile and critically has multi-touch input with a pencil and for apps that are designed for that hardware, things like Procreate and of course Muse, I think it’s uniquely good and it’s really special.

The other thing that I think people envisioned for the iPad was this new general purpose computing platform that would basically replace a lot of the things that the Mac desktop has previously done, and I never saw that and I still don’t see it. I think it’s a future we could get to if we all really want to, but I don’t see it happening right now. I know some people kind of use the iPad in that way, but I don’t get that at all. So we could talk more about that, but that’s what I see as the split vers on the iPad right now.

00:04:54 - Speaker 2: Yeah, well, speaking as a user, I’ve done for quite a while the big kind of stationary workstation, big monitor, mouse, I’ve got my big podcasting mic that’s set up on a boom arm. I’ve got various recording equipment, I got a ring light. This thing is not a mobile workstation.

At all, and I like that.

It allows me to have all these multimedia pieces that I need, but it also allows a more powerful computer with a bigger monitor and bigger input devices and so forth, as opposed to a clamshell laptop. And then when I’m traveling or going someplace, even in town for a meeting, I bring my iPad.

And this is just so much more portable, right? It’s not just the size, actually, it’s probably about the same in many ways as a standard MacBook, but in terms of battery life, instant on, I’ve got an LTE SIM card in there, which means it has always on internet, it’s really just truly remarkable as a portable device.

Now you do hit the limits of what it can do, and I run into that when I’m taking a longer trip if I’m traveling for a week, for example, and then I want to do something heavier, certainly anything to do.

With kind of web development, for example, but even editing a really long form essay or video editing, you can do all that, but you do run into limits.

There’s just less software available. The software that’s there is a little less powerful, but for me that bifurcated thing actually works really well, and I feel like the laptop is actually a weird mix in a way because it’s not as portable as the iPad, but it’s not as powerful as the workstation.

So that works for me. The idea of doing 100% of my work on the iPad seems untenable.

00:06:27 - Speaker 1: Adam, it’s so interesting that you and I have arrived at a totally different conclusions than this. I think that that’s been the case since day one. You were like, Mark, you should check out the iPad. It just feels magical. It feels like the future.

And my response was basically no, except for the pencil, which is awesome.

But you seem to really get along with it. I don’t know what to make of that. And I wonder kind of where the median or average user is.

I do think a lot of people get away with the iPad as a sort of laptop light, but I also think a lot of people, it just doesn’t work. And I don’t know, maybe that’s more evidence for the mixed verdict.

00:06:58 - Speaker 3: Yeah, I think for a lot of people, it’s a mix of use cases and they basically find the ones that work for them and to discount all the ones that don’t work for them. And that kind of helps the iPad in that, yeah, it often isn’t great because of the software as a general purpose computing was that really does everything you needed to do. And so as long as you’re fine with that and stick to the things that you know you can get out of it, then it can really fill that specific hole that you want it from.

00:07:25 - Speaker 2: That’s true. It may be in some ways the market, especially more on the consumer side, was trained in that direction from mobile devices generally and the iPhone, which is, I know a lot of people who, especially a younger generation or in some cases older people who they just always struggled with computers. Like desktop computers, the difference between I don’t know, minimizing a window and closing an application was endlessly confusing, file management, all this junk that never made that much sense to them. They don’t find it fun.

And so then along comes a mobile device where they can do 80% of what you can do in terms of sending emails and That kind of stuff and so they just try to do everything on the phone because the phone makes it easy, they understand everything. It’s hard to mess stuff up, you can’t get viruses, you don’t need to manage your files, and they just essentially decide to not do the things you need a computer for because it’s just they would rather be on the phone and then they can make the decision to.

Cut out some of those use cases, whereas maybe a really uncompromising user that has really specific needs, either niche software or just wants a lot of power, a lot of control, something like that is not going to be satisfied with anything but sort of maximum computing capability, and the idea of cutting out a few of those things that they can’t do is just sort of like untenable.

Yeah, that feels like the future point you mentioned there, Mark, is something that actually has come up a lot in our call user research, but basically just talking to people that use Muse or want to use Muse, which is they say something along the lines of the first time you use the iPad or when I open the iPad, it just feels like the future. It’s this magic device, it feels like they’re living in the future, and I certainly feel that as well, but in a way it’s sort of like a future that’s never quite coming true in the sense that you can do.

A lot with it, but again, at least when it comes to those creative tool things, they haven’t really made the jump, and it doesn’t feel like there’s a fast and furious, Adobe porting all their products over and except far superior versions, or what if you want to use Figma or sketch, those seem like really natural things that a person who is also the sort of person that wants to use the iPad as a creative tool would want, but you really can’t use them at all, and it doesn’t really seem like that’s gonna happen anytime soon. So yeah, again, it leaves this conflicted or mixed verdict in some ways.

00:09:42 - Speaker 3: Yeah, I’m always reminded of that first slide that Steve Jobs showed when he introduced the first iPad and it kind of had the iPhone on the left, the Mac on the right, and then the iPad was introduced as that third device in the middle.

And I think we’re still trying to figure out what exactly the role of that 3rd device is, even though we know, OK, it’s kind of supposed to be in between, but does that mean it takes things from the Mac and makes them simpler? Does it mean it takes what’s good about the iPhone and makes it better? What’s the actual use case that’s being solved by that 3rd device? Is it really only consumption based, which is kind of what a lot of people already use the iPhone for? or is there actually also a place for another productivity device or professional device that can do things that the Mac can’t do?

00:10:27 - Speaker 1: Yeah, I think this gets at the heart of the matter because I think once you put these 3 devices on one axis, the iPad’s already in a lot of trouble because the phone is already so pervasive and capable, people use it for a ton of stuff.

It’s great for content consumption, and even some creation now, and the Mac desktop is uniquely powerful, and now, I would say that they’re very portable, almost as portable as an iPad. So you really don’t have a lot of space left for the iPad in that model.

That’s where you get these like kind of marginal and incremental use cases like you have kids who use the iPad inside the desktops, it’s lighter and cheaper, and you have people who watch Netflix on the iPads as a bigger screen and like Adam types who take the iPad around so it’s a little bit lighter and more portable.

Yes, but it’s not fundamentally different in the way that the Mac desktop and the iPhone were.

Now I think there is a future where the iPad, it’s on its own axis, which is things like pencil, multi-touch, these things that are uniquely iPad. I just don’t see Apple really pressing on that front. I see that more from a few specific apps.

00:11:30 - Speaker 2: What might be good to talk about now our perspective as app developers in terms of a question that someone asked me recently that I thought was interesting to think about is what are the capabilities that you need from the platform to make your app better or more powerful or more professional.

And there are some things that could be surfaced as maybe APIs that we as developers can use to make our app behave in a different way, but a lot of it really does come down to the operating system, and so for me at least, I’d be curious to hear how you both see this, but for me, I think the operating system is the weak point.

The hardware is unbelievable, world class. I think it’s just the best computer we’ve ever made.

00:12:12 - Speaker 1: Oh yeah, the hardware is absolutely the best hardware that’s ever existed. It’s not even close, it’s definitely a software.

00:12:17 - Speaker 2: Right? And then the apps are weak, although I think a lot of that is Ato economics, and we inherited this whole iPhone consumer model and it makes it tricky to basically charge a reasonable prosumer price for your software, so that that’s holding it back as well a little bit.

But I think the operating system itself is one of the biggest weak points. And I was really excited when, what was it 2 years ago, something like that when they forked off iOS into, so now there’s iPad OS as its own thing with its own version number and its own that sort of thing. So I was really hoping that maybe that meant I have no idea what things are like internally at Apple, but there’s a team whose job is To make this operating system, it can diverge a bit from the phone. They did that in the beginning with the dock and drag and drop, which were both things that are only available on the iPad, and then that would allow it to find that unique identity instead of constantly inheriting things from the phone, which I think are at this point more of a liability than an asset. It doesn’t seem like that’s quite happened.

Yeah, and I’m curious, again, from the app developer’s perspective rather than say the user or just kind of market analysis perspective, how do we see our experience as an app developer and trying to make something for sort of professional use on this platform?

00:13:33 - Speaker 1: Yeah, well, let me give a quick list of what I see as the biggest features, and then I’ll build a little theory around that. So I would say it’s powerful multitasking, general purpose file management, payment structures and expectations, the whole payment situation, and more control over the run time in the form of basically downloading and running code things like scripting, extensions, plug-ins, and so forth.

These are kind of the defining features of a desktop operating system. And I think that’s not coincidental. I think the fundamental tension here is pro use cases are almost by definition about taking multiple different pieces and recombining them in novel ways that weren’t anticipated by the original authors of those pieces, because if something was simple enough to do in a fully premeditated and pre-existing package, it kind of almost by definition wouldn’t be a pro use case, right? That’s kind of a casual thing that’s already been done before. So you need this ability to recombine pieces in ways that weren’t anticipated by the platform, but that’s kind of antithetical to how Apple thinks about the iOS ecosystem of they want everything to be curated and controlled and to be on the rails, which they have many good reasons for, but that’s what I see as the fundamental list and perhaps a theory for why they’re not making a ton of progress on it.

One other thing I would add, I mean, I think you get very far with those things. I think if you want to realize the full vision of this third type of computer, you would need a lot of work on input. So an obvious thing is to have a bigger screen, more like a desktop size screen that you can put on your desk, multiple pencils, other physical input devices, and software that really took advantage of the 10 finger capabilities. Right now, most apps. Basically have one finger at a time. You have some apps like Muse where you can use multiple fingers, but you can imagine the muse approach to touch, which is use all 10 fingers being pervasive throughout the operating system in all apps and perhaps finding a way to replace the incredible speed and precision of a keyboard. That’s a hard problem. But I think you would need to tackle some of that if you want to really realize this third type of pro platform.

00:15:30 - Speaker 2: Yes, so from that list, it gets the programmability, the run time element, and that’s both individuals being able to write their own stuff, scripting or write their own little mini apps right on the device, that sort of thing, as well as something like plug-ins that basically are fairly strictly disallowed, and I do really see the tension there with essentially the security, you know, the App Store and the iOS and the mobile model Android has a version of this as well, maybe not as well done, but strict sandboxing, a little bit of a curation review process, and then just really kind of controlling what you can do. That is actually a lot of the reason the platform is good and is able to, yeah, your system isn’t bogged down by some weird ghost.

Process malware is not a problem, which is partially the programmability. It’s also partially things like runaway background processes and stuff like that.

So because the operating system controls all that so strictly, for example, a lot of that has to do with how the battery life can be better because the operating system has very, very strict guardrails for exactly what can run and when.

And so I think a lot of that is good and some of them may need to be changed or relaxed if there is pro use cases, but even before getting into that, I really wonder if there isn’t lower hanging fruits in the form of some of the other stuff on your list, and to me, a huge one there would be multitasking, and I see that two forms. One is just the interface, and happy to say that iPad OS 15 does improve on that a bit, but it’s still could be a lot better. It’s pretty awkward, basically, to like get two documents or two. Apps side by side and copy paste between them, and there’s things with focus on the keyboard and all that sort of stuff that is just not very nice, it’s not very fluid, it’s not very memorable, it’s not very discoverable, either for, let’s say a less sophisticated user or for a pro user that’s really willing to invest, sort of it ends up being maybe a clumsy middle ground, I’m not sure exactly, but I think that can be improved on from the app developer perspective, the harder thing is something like, yeah, for example, this background process thing.

So Muse we run into this a lot when we need to deal with a large data export or import or something like that. And so maybe if you want to export your entire Muse corpus, for example, in flat files, if you have a big one like I do, many, many gigabytes, that can take a few minutes. And I would just leave it running, except, of course, the device goes to sleep. If I switch away from the app, the process gets shut down after 5 seconds. Again, the operating system is very strict about how it controls that, which is part of what makes it good, but it’s also holds you back from these pro cases and we end up having to come up with all kinds of weird workarounds in order to do these things that we need to do.

00:18:13 - Speaker 1: I’m smiling over here cause I’ve long given the team a hard time about multitasking when Adam first said the iPad is the future. I’m like, is the future you can run one program at a time. Now, fortunately they’ve gotten a lot better about it, but yeah, that seems like an obvious one to get to improve.

One related thing that we talked about in the podcast before, and then I’ll bring up again is this idea of kind of a technology frontier. So right now with our current sandboxing technology, you do have these sort of two choices of the wild west and viruses and out of control processes and all your battery and an app store where you can’t have plug-ins and extensions and everything is very controlled.

Now, I think there’s a world where you have better sandboxing technology that allows you to get more.

Of those benefits at the same time. You know, for example, if you had much more granular and accurate accounting of what bits we’re using, what pieces of power, you could finally control that or whatever, right, while still allowing good actors to do some work in the background and shutting off all the bad ones, right? Probably actually the easiest thing there would be on the payments front where all the things that we need to do with payments are well known. And I think people would be fine using Apple if like you could give refunds and stuff, right? And there’s a whole series of things that we could do to make that pretty good. And that’s the kind of work and research that I would like to see Apple doing if they’re serious about turning the iPad into a new pro platform.

00:19:32 - Speaker 2: And what’s your perspective as a designer of the app? Are there places where you’ve found either huge benefits from the platform compared to, say, designing for the web or for desktop computer or weak points in terms of things you can and can’t do?

00:19:47 - Speaker 3: Yeah, I think to me the most interesting part is actually not even iPadres, like it seems to me that, OK, they’re kind of trying to make it more like the Mac and they are borrowing features from the Mac, trying to come up with ways to make them work with touch and this whole iPad system.

And eventually they’ll probably get there, like they’ll probably year after year, figure out more things to add and they’ll have more and more features.

Developers will be able to make more and more powerful apps.

But from a design standpoint, the more interesting question to me is what should these apps actually look like and what kinds of apps does Apple want us to build basically? And today, I really feel like it’s not enough for Apple to improve iPad OS. They kind of need to lead by example and build their own pro iPad apps and really have a shining light of an iPad app that shows everything that the iPad can do and shows the kind of interface that an iPad app should have in the minds of Apple designers.

00:20:47 - Speaker 2: Right, so one thing Apple could do if they really wanted to lead the way on the design front would be to take their first party apps, keynote, numbers, pages and use them to really demonstrate not just hey, here’s a reasonably good port of a Mac app to the iPad and it’s usable, but actually really go above and beyond and make it something where imagine Keynote seems actually like a pretty obvious example of something that’s fairly visual and tactile. Could you make it so that the keynote experience which so much better. People really preferred doing it on the iPad to the Mac or the spreadsheet actually is another interesting example where not only is that such a venerable and useful kind of staple productivity tool, but also to me it feels like pretty natural on the tablet form factor, and I often am poring over spreadsheets with I don’t know business financial models or something like that, and it’s nice to sit back in that more ruminating posture in the reading chair and what have you, but beyond just kind of.

Assuming or very minor changes to a spreadsheet is no fun at all to do anything with a spreadsheet on a tablet.

I feel like I could picture just maybe more emotionally, I can picture what it would be like to have a spreadsheet that’s really amazing and fun to manipulate on a tablet, even if it was not as powerful, but maybe for like the very most basic common operations that you do that it could really showcase that form factor’s capabilities, and yeah, no one’s led the way on that, not Apple.

00:22:15 - Speaker 3: Yeah, and I think there are a few layers to it, right? So one reason why I actually want Apple to build more of their own pro apps for the iPad is that I think that will make them see sort of the pain points or the gaps in the iOS iPad or interface. So I think a lot of the difficulties that we have with Pro iPad apps are actually because of gaps in the iPad OS.

00:22:39 - Speaker 2: So maybe if Apple was putting more effort into its first party apps less because they want to be successful with those apps and more as a showcase or an example of what this platform could do, then in turn they would be exposed to the weak points in the platform, things that the app developers need like background processes or more powerful gestures or other things they’ll discover those and then in turn the platform would get better. But that sort of begs the question also of What does make a great pro app or what does make a great pro app on the tablet?

00:23:12 - Speaker 3: Yeah, and one way is certainly to just look at the Mac and see what’s working there, try to bring that to the iPad.

And I think sometimes that works and that’s what Apple has been doing.

So for example, I think in one of the recent versions of iPad where they’ve tried to bring the right click from the Mac to the iPad. And so since the Mac has a mouse with two buttons, you can have a right click. The iPad doesn’t really have that. So instead, you have the long press on the iPad, basically. And then you get the same sort of context menu that you would get on the Mac, which works. It does sort of add another layer of more options that you can add like some hidden complexity that you didn’t have before. It’s basically the same thing as on the Mac, but it’s just a worse version of it.

00:23:55 - Speaker 1: Yeah, I agree there’s a lot of stuff that you can transliterate over from the Mac, and I’ve argued earlier that they basically should things like multitasking and a real file system and so on, but to your point, it’s only gonna get you kind of 80 to 90% there because a desktop will always have a bigger screen or will have a keyboard, which is an incredible input device. Again, I think if you want something really interesting here, you need to take advantage of the things that are unique about the iPad, which are the pencil and 10 finger input, and I just don’t see a lot of activity there outside of a few apps right now.

00:24:28 - Speaker 2: Now, since it seems like we’re falling a bit more on the negative side, let me balance that out with a bit of positivity. Also, since I’m the iPad feels like future guy, whether or not that feeling is correct or not.

One example is the pointer stuff they introduced last year. So this is essentially if you have a trackpad or a Bluetooth mouse connected to your iPad, you get this little translucent circle that is your mouse cursor effectively, and it sort of morphs according to what it’s over.

So, for example, if it’s over a button or if it’s over an app, it’ll turn into a rectangle shape that mirrors what it’s over, and this Sounds like a pretty minor thing, but once I used it, now going to a desktop and it’s mouse cursor feels very old fashioned, and it actually kind of boggles my mind a little bit that something so important and basic as your cursor, which you’re looking at all day, you need to spot it on the screen, you use it to do everything, basically hasn’t changed in, I don’t know, 25 years.

And not to say that things need to change all the time, but generally that’s a good indicator in the technology world that we’re improving computers and they change and grow with time.

And just seeing this in some ways kind of minor design tweaks on what the pointer can be, but it feels better, it looks better, it’s more functional, it’s more discoverable, and I just go, wow, this is great. Like, can we take more of these basic sort of. primitives and apply some new thinking to them and things you couldn’t do before, right, these smooth morphing animations, even something like a translucent cursor, was not possible at the time these black and white cursors that Windows and Mac and Linux use. Translucency was like a high powered graphics operation. No way it could be a part of your standard mouse cursor. Today, of course, that’s totally a trivial thing to do.

Now, a counterpoint there might be people are disappointed that they are not applying this sort of innovation to the Mac and are investing it in the iPad and in fact, the Mac is the work and productivity platform. Why not improve something like pointing devices there instead? I find the contrast really interesting, especially for someone like me who goes back and forth between a Mac and an iPad in my daily work.

00:26:43 - Speaker 3: It seems to me like one of the most exciting parts of the whole iPad platform, or at least the iPad system, is that Apple does have teams like that that like they probably spent years just designing and developing this cursor system and getting all the details right and really going back to the start and not trying to just take what’s on the Mac and kind of apply it to the iPad and make it work somehow, but really think deeply about what its place on the iPad should be.

00:27:11 - Speaker 2: Yeah, I see that as the positive version of transliteration.

I feel like this happens in personal life.

For example, you move to a new house or a new office, or maybe if you’re changing your productivity tools, it sort of forces you to take stock of all the stuff that’s there.

You had weird stuff hiding in your closets or you hadn’t really rethought how your kitchen was arranged because, you know, you just had what you had and it worked fine, but when you’re changing it.

Everything, then you stop and you go, wait a minute.

I’ve actually changed my cooking habits. Let me change my kitchen to match that, and you actually can end up with something much better.

And so I think there is a version of this where we’re translating things like keyboard shortcuts or right click context menus or mouse cursors to this new platform, and they think, OK, well while we’re here, let’s rethink it.

Let’s take the things that are really great about it. And actually even keyboard shortcuts is a good example of this to me, like this kind of system-wide default capability affordance on the iPad, which is when you hold down the command key, you get a nice pop up that shows you all the currently available keyboard shortcuts in the current context in a format that’s really standardized. You know, most Mac apps have some kind of keyboard shortcut help sheet, but it’s hard to find, not everyone has it, and it’s just always right there, and it’s incredibly discoverable because if you hold down the command key and you’re like, hmm, wait, what do I want? And you kind of pause there for a little bit, then it pops up because you’re sort of being indecisive. I think it’s like a really nice example of bringing across keyboard shortcuts are amazing, including modifier key-based shortcuts, but bringing them to this new platform was a chance to improve and enhance. And so I see that as a lot of the ways in which there’s big potential in the iPad, and what we just don’t know is whether that potential will be fulfilled.

00:28:53 - Speaker 3: And notably both of those innovations are about inputs to the iPad and accessories to the iPad, the, the trackpad and the keyboard. And to me, that’s really what’s most interesting about the iPad and when you compare to like the Mac or the iPhone. We say the Mac has a keyboard and it has some sort of mouse, and you can kind of guarantee that every Mac has that, and that’s not really going to change. Windows has some touch stuff, but that’s more added on top like no app really makes that great of a use of it. And the iPhone just has touch and they aren’t showing signs of trying to add a pencil or external BlackBerry like keyboard to it, right? Versus the iPad has really this flexible system of inputs, by default, it is touch and that’s sort of the basis, but then every user adds their own input devices to it. Some add an external keyboard to it, some have a keyboard case where it’s semi permanently attached, and then you have different kinds of pencils.

00:29:51 - Speaker 1: Now I’m realizing as you two describe all the different ways that you use the iPad, it’s kind of alarming because unlike the phone, And the desktop, where there’s basically one way to use them and the phone it’s the thumb or the pointer finger, and the desktop it’s a keyboard and mouse.

It sounds like people are using the iPad in all kinds of different ways.

I could come up with at least 4.

There’s the muse style, 10 fingers, there’s the you’re holding it with one hand and using a 1 pointer finger. There’s the Adam Wiggins style keyboard with the iPad propped up.

And there’s maybe it’s lying on a desk it’s a 4th way, right, with a pointer or a mouse like device, and it’s a benefit because there’s always different ways that you can engage with the device, then as an app designer, you kind of don’t know how they’re approaching the app, and I guess in use cases we’ve kind of had to say we’re gonna embrace this one or two styles of using it where, you know, for example, we kind of assume that you have a pencil, but I don’t know, maybe if the different input modes proliferate that becomes a sort of bigger problem.

00:30:47 - Speaker 2: Multimodal input is, I think, one of the things that makes the iPad, or maybe just the tablet form factor generally, the most exciting to me. I agree it’s a huge design problem as well as just user research problem. You can’t necessarily support every possible combination that people have, but I think that that Reflects how computing is changing for humanity overall. I mean, it wasn’t that long ago when you wanted to use a computer, you would go into the room where the computer was, you would turn it on and wait for it to boot up, which took a couple of minutes, and you sit down and you start your computing session and you do that for some length of time, 20 minutes, an hour, whatever it is. When you’re done, you power the whole thing down, you stand up and you walk away, right? And then mobile brought this thing where it was so integrated to our daily lives. You pull out your phone, you look something up really quick on the map, or answer a text message or something, pocket your phone again, and now comes, I think I read somewhere, some statistics of people look at their phone 100 to 200 times a day, pretty commonly, and some of that maybe is social media engagement loops sending you breaking news, notifications. really need to be looking at your phone and you can talk about all the ways that that’s interrupting, I don’t know, more human conversations and whatever, but putting that whole discussion aside, I think that this thing where computing is woven into our daily lives, where if I just want to Google something quickly or look up the hours in a restaurant or pull up a note on something, I can do that quickly and return to what I’m doing in context. I use that all the time from everything from looking up something with one hand while I’ve got my baby in the other hand, you know, when you’re out in the world, all that sort of thing, and I really like that, and the mobile platforms powered that.

And so I think the iPad and thinking of the iPad in again more of a work productivity setting, it’s less about just whip it out and do something quickly in 5 seconds and put it back in your bag, and more than I’m here in my office, and I’ve got the iPad with the touch capability, but it’s also got this really nice hardware keyboard. I’ve got the trackpad, I’ve got the stylus, I’ve got voice input. I use the dictation. Not hugely, but sometimes I’ve got my AirPods and I can listen to things. So basically there’s all these different ways I can interact with it. I’m moving around the room, I may carry it into another room if I’m in a meeting with someone, and the laptop, I think, kind of for all its mobility, it inherited that desktop. You sit down and you’re in one posture, and that’s sort of the position you’re in, and it’s this integrated to life. And that’s sort of related to or overlapping with the multimodal input.

For me, it’s just a much more creative, comfortable, fun, I don’t know, it’s just like, once you’re there, you can’t go back, but then you have to go back because you can actually do most of the things you wanna do on this platform.

00:33:36 - Speaker 1: No, totally, that to me was the magic of the iPad. It wasn’t the cursors for me, Adam, but it was the direct manipulation of the iPad with my hands. It just felt so human in a way that the computers and even the phone never did. So yeah, plus one on leaning into that for the future of the iPad.

00:33:58 - Speaker 3: I think this is another case where the iPad software lags behind the iPad hard, where you have all these different input devices. You have touched on the iPad, the pencil, the keyboard, draws a trackpad things, and you can really mix and match them. You can use the pencil in one second and switch to the keyboard in another, and it all works great.

But then on the software side, they still kind of feel like different modes. When you use the keyboard, you are probably editing a text field somewhere. When you use the pencil, you’re on some sort of canvas sketching area.

And as soon as you go outside of that, the pencil only emulates touch, basically, like it doesn’t add anything to the experience. So that’s why I would hope that Apple advances iPad OS in a way that you can really combine these and say, press a key on your keyboard while touching something or while doing something with a pencil. And that’s also why I think it’s important that they start building their own pro iPad apps because in the end, that doesn’t only need to be reflected in the system software, but also in every app and you just kind of need to come. To expect how these different devices that you can use with the iPad really interoperate and not just uh stand for different modes.

00:35:08 - Speaker 1: Yeah, I think this is an incredibly rich area. I really hope we see more work in this, and I would emphasize that it’s a lot of work. Like we have a multi-year research program going through ink and Switch, and now Muse, like, how do you use more than one finger at a time, right? Just that alone is a huge deal and doing it in a way that’s responsive and accurate and so on. And so I could imagine teams working on this for years to really bring that vision to life.

00:35:34 - Speaker 2: The one thing that often comes up when folks are talking about the future of the iPad is whether it will or whether it should merge with Mac OS.

So there’s something that happened a little bit in the Windows world, for example, the Microsoft Surface hardware, which is one of my favorite tablet stylus form factor hardware pieces.

But of course it runs Windows with all the baggage that entails, and they have found ways to merge the touch and the stylus and the mouse cursor that I think are not entirely successful, but you see where they sort of brought together those platforms and those paradigms in their way.

And many have argued that Apple is doing something similar.

They’re on a long, slow progression whereby, for example, adding things like trackpad support to the iPad or you look at something like the control center in Mac OS Big Sur and has these very big kind of touchable chunky things that look like you should touch them with your finger, but In fact, of course you can’t because the Mac doesn’t have a touch screen, but at the same time, I think Apple’s been publicly on record saying no, we’re not planning to merge those together, so I’ll put the question to both of you, do you expect that as a thing that will happen? And then separately from that is the thing you would like to happen or that you think it’s a good idea?

00:36:50 - Speaker 3: So to me, it kind of comes back to the question of what Apple wants the iPad to be and what really is the core of the iPad.

And there are sort of a few possibilities there and it kind of worries me that we still don’t know what it is.

So one possibility is that it’s really about the simplicity of the US as we talked about that it just has more restrictions and it’s just something that is a simpler version of what the Mac does. And in that case, I don’t think it can replace the Mac. Then they are clearly positioning the iPad as something that is more approachable and less complex as the Mac, so the Mac has to stay where it is. Although then I would also argue that Apple could invest into the Mac a lot more and actually go into the opposite direction with the Mac and make it a lot more complex and say if you don’t like that, you can always go to the iPad.

00:37:37 - Speaker 1: Yeah, I see it similarly, I would break this into two questions, which is, does the iPad grow to support pro use cases, which means things like really powerful multitasking, powerful file system, run your own code, things like that, as well as all the input stuff that we talked about. That’s question one.

And question two is, does Apple want to continue to support pro users? So there’s a world where the iPad continues to not support pro use cases, and then kind of part B of that is Apple could continue to support pro users through the Mac, or it could basically sunset the Mac and say, you know, those folks are cool, but it’s a relatively small piece of the market. We’ll let Windows and Linux deal with the weird like audio editors and stuff and other normal people can use iPads and iPhones.

My bet for the first piece is that I would love to see them turn the iPad into a Pro Tool, as we’ve talked about, that’s a huge amount of work, it’s a long path, so it’s kind of hard to predict that they will do that. It’s kind of hard to imagine them giving up on. The Mac because those users are such a keystone piece of the ecosystem, among other things, it’s all software developers. That would seem to be a mistake to me, but who knows, maybe there’s just so much money in the iPhone, the iPad that they can get away with it. But I don’t think that they will do is they won’t get pro users to use a non-pro tool. Just won’t happen. People use our platform as they have in the past.

To be clear, the future that I want to see for the iPad here is that they make a 24 to 30 inch version that has all of these powerful features and that can replace or appear to the Mac desktop. I think they could do that if they want to. It’s just given how things are going, it’s hard to predict they will do that at this time.

00:39:10 - Speaker 3: Yeah, I think that’s a really important bridge basically that Apple could cost.

Right now, yeah, iPad is the super mobile device and everything about it, including the different inputs you can use with it.

I kind of built towards. OK, you can hold it in your hand, you can have it at your desk, you can have it on your couch, and you can switch between those within seconds.

And I think that’s part of the really big difference that right now iPad is a really mobile first device and I could very well see Apple deciding, OK, that is what iPad is about and we don’t want to make it a 24 inch or even like a 20 inch, 16 inch device because then you can’t really hold it in your hand anymore. And that’s really like the line we draw between the iPad and the Mac.

00:39:50 - Speaker 1: I also think, by the way, if this gets a little bit beyond the iPad, but I think if Apple chooses not to pursue this future of a pro tool for the touch surface class of devices that someone else could do it. So, you know, someone could go buy a 30 inch touch screen. Those are becoming increasingly available and write the software and plug it into Windows or something. I guess we’ve kind of seen Microsoft try that a little bit with their line of what’s that called the Surface hub, surface, yeah. I think it would be a real shame if that future wasn’t pursued somehow, so if Apple chooses not to do it, hopefully we’ll find another way.

00:40:25 - Speaker 3: And certainly I think it would help Apple embrace sort of the general purpose nature of the operating system, because it doesn’t make sense on a 24 inch screen to have a single device, and it doesn’t make sense anymore to use it on the couch. So you want to have it on a desk and you want to do things on it that you do on your desk, which are naturally more complex interactions. So in that way, I think it would be really exciting for Apple to build a larger iPad, even for the people that don’t want a larger iPad, like they would probably still benefit from the development that the iPad gets out of it.

00:40:59 - Speaker 1: There’s an incredible endgame here where what was originally iOS becomes adaptable from the phone to the iPad to a Pro desktop class tool, and if you were able to figure that out, if you were able to succeed in that research project, you could have this incredible fluidity between the devices, maybe even using the devices together, for example, your iPhone is on your desk as a little sidecar with some extra controls while you’re working on your main iPad plus.

00:41:27 - Speaker 2: Yeah, I love that. Maybe that’s extending the multimodal input even a little further, as sort of a multi-device world, which I think we already kind of live in, you know, you’ve got your Kindle, you’ve got your fitness watch, you’ve got your computer, there’s other kind of devices that float around in your home or your office. And I always like this kind of Hollywood thing with uh Tony Stark on his lab where he’s got his like robot assistant he talks to, but he usually has multiple screens and this is basically just a Hollywood thing, but in some ways it also is compelling that the room is the computer and the screens and the different devices, whether they’re touch screens or holographic displays or voice interfaces, they’re all just different affordances into that same computing medium.

And I think in a way, we kind of have a version of that now, in the sense that we do have lots of devices floating around on our desks and in our homes and so on, but they don’t coordinate that well with each other, so yeah, you can imagine that there is a version of iOS that flows across all of those different size screens and different form factors, and they work seamlessly together, that could be pretty cool.

00:42:35 - Speaker 1: It’s so rare that when it does happen, it’s such a shock. I remember the first time I experienced the Wi Fi flow where you try to log on on one device and it like another device that has the login, sends it to the other device. Oh my god, that’s so cool, right? But you can imagine that for everything. Yeah.

00:42:52 - Speaker 3: Yeah, and I feel like the fact that Apple is putting that many resources and making the devices, the Mac, the iPad, and the iPhone work together really points to me more to the fact that, yeah, they aren’t trying to replace the Mac with the iPad. Like they are seeing the iPad as a 3rd device and they want you to use it even right next to your Mac and like they showed, I think with the iPad OS 15 and the next Mac OS version, you can kind of use your MacBook trackpad and actually move the cursor over to the iPad and then control your iPad with it and also use your MacBook keyboard. And it really, at least from the demo, it really seemed seamless.

And to me, that’s really the exciting part of what Apple is doing, where if the iPad is a 3rd device that I’m supposed to use next to my Mac, then they can actually figure out these specific use cases that the iPad is good for, and they aren’t forced to bring down everything that the Mac does to the iPad, but they can say, OK, you you have the Mac, you also have an iPad. And we can figure out exactly what interface works best for each of those and maybe even more importantly, which use cases are best for each of those.

00:43:53 - Speaker 2: Now obviously here we’ve spent plenty of time speculating about what Apple will do, what they should do, what their opportunities are, and that’s, I think a lot of folks in the industry because they are such a powerful player, and certainly anyone who is an app developer, you’re necessarily very much playing their game, and so what Apple, who never, you know, announces ahead of time their roadmap or their intentions, where they’re going, becomes a source of maybe endless speculation.

But I think it’s useful sometimes to stop and just think, OK, separately from what Apple will or won’t do, what is the computing future that we want? We got to this a little bit with Rasmus Sanderson and that episode talking about some of his vision for Playbit, but notably here at Ink and Switch, Mark, you and I worked together along with a bunch of other great folks on various research projects, and in a way, we saw them circling potentially a larger vision.

I think at the time we called it the programmable personal knowledge manipulator, not that catchy, I suppose, but, you know, you gotta start somewhere.

And we envisioned something that had a form factor similar to an iPad or a surface, where you have the tablet and the stylus and the 10 fingers for touch, but potentially other kinds of input had maybe, you know, local first storage, so you have a powerful file management like you do on the desktop, and more suitable for collaboration in this sort of cloud world, and that furthermore, it’s fully programmable, and then maybe the base device doesn’t. Do a lot. It doesn’t come with a lot of apps, it doesn’t have a lot of features, but you could sort of write your own apps and browse the web and sketch, and that something like that could be a very fun and powerful new kind of pro platform.

Again, not necessarily trying to replace the desktop, but a way to take these computing capabilities that we have with modern hardware and everything that’s been pushed forward by the mobile revolution and bring that to the creative tools space. And we even put some work into trying to bring those pieces together into a prototype, but we actually determined it was just too early, too hard, probably too big for any one company to do.

So that’s part of where we kind of split out the different pieces, and one of those was Muse. We said, look, the best way to explore this kind of multimedia canvas side of things is on an existing platform and that platform was the iPad. But I still have that shining vision floating in the back of my head, and I think it both leads me to, I’m doing the mental diff between where the iPad seems to be going and that vision that I have for that programmable knowledge manipulator that I want. And the ways in which the iPad is changing to be more on that trajectory versus not, makes me happier or less happy with the iPad, but then maybe separately, like you said earlier, Mark, maybe someone else needs to build that, and it’s a huge undertaking, but maybe Apple isn’t the right company. Maybe they’re a consumer company now, not a creative tools company, and maybe something, another company or another team or a set of companies or open source project, I don’t even know, that could really be focused on that audience and that sort of set of use cases could do something pretty special.

00:47:00 - Speaker 1: Yeah, and I think it’s interesting because there don’t seem to be a lot of people really working on this. This is problems like the multimodal input problem, including with 10 fingers, the sandboxing and security problem while maintaining power and flexibility. There is not a ton of work on this that I know of. And so the flip side of that is that if you do get a small group together and work on it for a few years, you can pierce the frontier, you know, you can make a contribution to the field. So I’d love to see more people try that.

00:47:28 - Speaker 3: Yeah, and I think it’s kind of where change needs to come from. I think Apple does have like a ton of really good research groups that do this kind of research, but it all stays within Apple and they’re famously secretive. They certainly don’t show it, but a lot of it also either takes like 10 years to develop. I think a lot of stuff like even the MacBook Touchar that nobody really likes took like 10 years to develop from like the first research stuff. But like 90% of what they come up with will never see the light of the world simply because.

Apple with the iPad and especially because it’s based on the iPhone, it’s now at a point where it is such a popular and widely used device that they can’t really change anything fundamental. The only thing they can do even with the iPad is to add stuff on top of it, which might improve things somewhat, but they will never really be able to change the game. And so what I would really hope for is, yeah, we basically need some sort of newcomer that doesn’t have any legacy to worry about. And can really just start fresh, but that gets more difficult with every year basically because there’s so many more things you need to do and the ecosystem that Apple and Microsoft and Google have just grows bigger and you can’t really compete with it.

00:48:42 - Speaker 1: Yeah, it is tough. It’s why I think this idea of research prototypes was so important out of the lab. You need something that’s higher fidelity and more information than just like theorizing about something and drawing some sketches, but to turn it into a production product that’s integrated with an existing platform is an enormous amount of work we’ve seen with Ms even to do a tiny slice of it, as many years, right? But these research prototypes, they’re real software, they’re working, you can play with them, but they focus on one or two dimensions. And so that’s perhaps a way to tackle that.

00:49:14 - Speaker 2: Well, before we go, coming back to the iPad, if you had a wish list item, a genie that could grant one wish for something that could be added, some major change to the iPad as a platform, say, 3 years out, what would that be? My answer to that would be developer tools. There was a great terminal and the ability to write your own apps directly on the iPad and run your own apps and possibly even give them to your friends.

And finding some way to resolve that problem of you want that sandbox security, and you want the app Store curation that protects against the wild world of difficult malware, but at the same time gives you the freedom and flexibility to program your own computer, and I think that that in turn would kind of solve. A lot of the other problems, because then the developers could start to do more of the innovation and discover more weird interesting use cases. If they could do that and the thing that is not constrained by Apple review because it’s just for yourself and a couple of friends, then I think some very interesting things might emerge from that that could then solve a lot of the other problems.

00:50:21 - Speaker 3: For me, I think it would be text selection. It’s sort of the underlying cause of so many small frustrations that I have when using the iPad. And basically, whenever you work with text, you kind of need to select things and move the curse and it naturally doesn’t really work with touch. And so either Apple needs to figure out a way to just make it work more precisely with Touch, or maybe even leverage all the input devices they have and make better use of the pencil and the keyboard and just let me use those in combination with touch to accurately select text.

00:50:52 - Speaker 2: Yeah, probably that area of things might even be bad enough that just declaring bankruptcy just completely remove everything with the current touch base text selection, which is just doesn’t work well, has never worked well, and instead start over from scratch, and maybe that’s, you can’t even select text at all with touch and you need some other input device, or maybe they just have some wild new idea for how to do that. But yeah, what’s there now is not good.

00:51:19 - Speaker 1: Well, it’s tough for me to pick just one, but a very practical item is multitasking, and there’s a very simple test here, which is the multitasking needs to be good enough for me not to be so mad that I agitate for us to write our own in-app multitasking in use. We’re still not there yet, but I believe we can do it and thereby avoid a bunch of work on our part.

00:51:43 - Speaker 2: Well, let’s wrap it there. Thanks everyone for listening.

If you have feedback, write us on Twitter at @museapphq. You can reach us on email at hello@museapp.com, and you can help us by leaving a review on Apple Podcasts.

And while we’ve had our gripes and concerns about what the iPad is today, where it might be going, I think clearly the fact that we’ve all chosen to devote our careers here in the moment to building exclusively for this platform means we see its potential, that it’s one of the most interesting. Fast evolving places in computing right now and certainly for building thinking tools it offers new capabilities that I think are not available anywhere else. So I hope you both still feel positively about the potential for the iPad because well, you’re betting your day job on it.

00:52:32 - Speaker 1: Absolutely we criticized because we care and we love the platform.