Ep2 – Is Self-Hosted AEM Still a Thing?

[00:00:10] Speaker A: All right, so here we have our second edition of this podcast. And I am joined here by Joey Smith from Rightpoint. He's the Senior Director of Engineering at Rightpoint. I'm Tad Reeves, am architect or some sort of individual like that, digital. And we're going to be talking today about Adobe Experience Manager and whether or not Adobe Experience Manager self hosted or hosted not at Adobe is still a thing and whether or not it should be a. So I guess we can just kind of just dig right into this. So first of all, so super high level on Adobe Experience Manager. I think I want to just kind of talk through what it is, what it does, and why somebody should want to know about this at all. So I've got a little simplistic diagram here. And Joe, do you want to just give a quick overview on the guts of this, but also why any of this matters to anybody? Sure. [00:01:21] Speaker B: So at a high level, AEM or Adobe Experience Manager consists of these three layers that you're showing right here. You've got what's called an Author, a Publish or publisher, and a Dispatcher. So these three pieces are used by or enable different parts of the Web content management experience. Actually, I think one of the neater parts, or what I think is one of the better architected parts of Adobe Experience Manager is this idea that we've separated out Authoring experience from the end user experience to this day when I still have to go back and occasionally work in somebody's WordPress or something. And it just horrifies me. I'm like, why is the content management part of the user experience? It just doesn't make sense. So in AEM, the author and the Publish instance are both built by the same Java program. So it's a jar that's shipped from Adobe that depending on how you started, it takes what's called a run mode to say, am I the content management piece or am I delivering content to an end user? And then because it is Java and because the nature of how an AEM page is built out of a bunch of dynamic components, they also tend to put it behind what they call a Dispatcher, which is a proprietary module for the Apache Httpd. That is a reverse caching proxy with kind of a Rudimentary WAF or Web application firewall built into it. So you can say certain parts of the application are firewalled off or only accessible from certain clients or certain IP addresses and things of that nature. And then as people start to browse the website, it's building up an on disk cache. And the Publish instance and the Author instance have this ability to go out and talk to that same Dispatcher module and say, please invalidate the cache that you have on disk. I have newer content. There's also rules you can do things like time based and things like that. But the general concept is that you have this separation of concerns. I've got the author instance which is where my employees, my content creation team goes in to make new content in a component based way. The publish instance that is a rendering agent or this job is just to render pages. And the dispatcher, which is a Caching proxy server so that people aren't constantly getting every page rebuilt from scratch, we're able to say, you know what, this page is cacheable, let's go ahead and serve that out of the cache. [00:03:55] Speaker A: It's awesome. And I think that the dispatcher and its flexibility, its simplicity and the fact that you can have tons of them is one of the things that for the last decade plus, has been what makes Am so attractive to a lot of big companies with potentially tons of traffic is that you can be super flexible about how that is delivered and how much of that is always going to be dynamic and how much of that is cacheable in a really high performance server. [00:04:25] Speaker B: Yeah, for sure. It used to be a thing my friend Pete and I, who also works at Right Point, we used to kind of tour the country talking to AEM user groups to say, boy, if you're not building your content so that it can be cached, you're missing the boat. On AEM developer. The title of that was Develop with the dispatcher in mind. I think you can probably still find that podcast out on the Internet if you Google for it. [00:04:48] Speaker A: You were a prolific podcaster back in the day. [00:04:52] Speaker B: We used to do a lot more. This is one of those things where the day job gets in the way and the content creation isn't part of what I do for my paycheck. [00:05:02] Speaker A: Yeah, okay, good. Well, yeah. And that caching too is one of these things that when it's very easily fiddled with by the development crew, it's one of these things that you can have a lot of business confidence in. Also because I've been on teams and in businesses where they had had a lot of problems with caches being invalidated at higher levels and at your content distribution network and so forth, where you say, hey, I put out new content but it's still not being shown. When you have the ability to really, finally control and instantly control that Caching tier, then it makes for a lot. [00:05:37] Speaker B: Of and to enable the content creation team right, to control that Caching so I can give them the ability to flush the dispatcher in certain regions in a controlled manner. So that way they don't have to be dependent on me to wait to invalidate their cache. But I don't have to be worried about them going out. And invalidating the main corporate home page, and all of a sudden, all of our published instances go down because we have no valid cache anywhere. So it does allow that very fine grained control and you can say, you know what, we've got a few users that are trusted. They know the platform well enough that we're going to give them that ability. [00:06:15] Speaker A: I want to move on to one other diagram here and this gets more complicated, but I think that this might segue as to why people don't like doing this all by themselves anymore in a lot of cases, because it can get really complicated really fast. So one of the things I guess to bring up first is that the publisher and the author that you'd brought up are both Java application servers and they're historically not terribly simple to set up and definitely not simple to debug. The other thing too, and I'm not showing this on these diagrams because it'd get way too messy, way too fast, is that in every single implementation of this, it's never just sitting in a vacuum. It's not just itself. Almost always it's connected to something, usually many somethings on the internal corporate side, whether it's a legacy CMS or some sort of search tool or the commerce system or whatever it is. And it's got its fingers in all kinds of different things and there's all kinds of ways that can make things get vastly more complicated to run. But this diagram gives a little bit of the way that you might see it typically set up, or maybe two, three years ago would typically see it set up before people started having an option to have a cloud service to migrate this to. Go ahead. [00:07:45] Speaker B: Yeah. So I was just going to add that, as you were mentioning, there's a lot of when you get AEM, you're probably not going to say, you know what, we're just going to use this thing out of the box. It's not WordPress. Right. What it is is it's a platform for building an extensive content management that exactly matches your business rules and has all of your businesses integrations that you want in one place, as opposed to something that's maybe a little more simple and a little more direct, but doesn't have the extensiveness and the capabilities that AEM has. And one of the things that I should have mentioned earlier that I think a lot of people struggle with when they're first exposed to Adobe Experience Manager is it's not your traditional three tier architecture where you've got the application layer, the database, and then some kind of rendering. In AEM, the application state is stored in the embedded database, which is the Java content repository or JCR, and that is part of the application state. So the pages and things like that, you can't just say, oh, you know what, my database is going a little bit slow. Let me shard the database and add some more RDS or something like that to it. Because it is using this kind of in application database, it becomes a little more difficult to scale. You either have to scale vertically, meaning make our instances larger add more CPU, add more Ram, things of that nature, or you have some limited ability to scale horizontally. But that has licensing implications. You have to look at what you paid Adobe for. They've only allowed you to use a certain number of production instances and that can get pretty expensive pretty quickly. [00:09:30] Speaker A: That's right. And that's where you get really so what you said too, about how everything is in this Java content repository and the application state. Well, that application state includes all of the installed code, the running state of that code, and it includes every piece of content and what its current status is. Is it visible to the public? Is it there? But it's not visible to the public. How many revisions does it have? Everything about it, what users were created, what possibly external facing users were created. Yeah, exactly. So all these things is all in this Java content repository. And so what you can do, yes, you can. These published here servers that are the application server that renders this app to the public, you can go and copy paste those. But because of the fact that every single one of those has all of that data that it has to contain in order to be a complete server, those servers are gigantic. Sometimes the small ones, the little itty bitty ones are generally around 500 gigs, and the bigger ones can get much bigger. We're talking about multiple terabytes. And those are ones too, that you then say, well, can't we auto scale even? Let's just say licensing wasn't a problem because you generally pay for every single one of those that you have. Unless you have a site license, which is exorbitant, then you're paying for every single one of those application servers. So let's just say licensing was out of the picture and you can do it even still, if you have 1 server, you can't just copy paste that and there's no cloud anything that can make it go any faster. When you're saying, yeah, I am actually going to duplicate a five terabyte volume or a two terabyte volume and make a bunch more of these things. So it makes scaling and applying, even if you're hosting in the cloud, if you've moved it off of your on premise location and you're hosting in the cloud, it makes that really trying to because of the fact that all of that has to all be together in the same repository. [00:11:42] Speaker B: Yeah. And there are some tricks that as AEM matured a little bit, there are some kind of workarounds and different things that you can use. They have different backend engines that you can use and stuff like that. But even those all come with their own significant costs in both management and performance that there's never really been just kind of this magic wand that we can wave and say, oh my, AEM is now really scalable and I can auto scale on demand. Even the few customers I've worked with that did pay for that site license. They have to very carefully consider when are we going to actually bother to spin up a new publish instance? When are we going to actually resize our author instance, or maybe even consider sharding our author instance. There's so many risks and challenges that. [00:12:26] Speaker A: Come with all of that and the author instance too. And that's I think, a point that is going to take us on to just kind of what makes having a cloud service so cool is on the author instance, there has never for the history of this product and this product has been around since in general usage for whatever, 15 years or something like that, or more. And in the history of the product, there's never been a practical way to horizontally scale the product for most people. In some cases, there was the ability to use a mongo database behind it for very large, where you have more than, whatever, a couple hundred concurrent users, you could pay for the extra license that instead of using on disk storage, you're using a mongodatabase to store all the back end things. There was a performance implication to that too. It was mostly for high availability that you're doing that. But there's never really been a practical way. So let's just say you have a use case. Like you've got a big very image and video driven website with a lot of graphics and a lot of video, a lot of stuff. That where people are uploading big assets all the time. They do a video shoot and there's another 50 gig worth of video files that they're just slapping onto this thing that is then being transcoded and getting ready for viewing and so forth. All of that has to happen in one great big author computer that sometimes is like, whatever, 30 CPUs and 128 gig of Ram or something. There's no way to do it than this uber megatron author machine because there's simply no way to piece and part that practically, like you said, there's funny business you can do where you can offload some of those workflows. But for many people that wasn't practical. And there's a huge DevOps overhead then to trying to make any of that kind of stuff work. So I guess we can then take that to the cloud service because about three years ago so January 2020 is when this was unleashed upon the world. It was hinted upon earlier at the Adapt To conference, but it was publicly unleashed upon the world in 2020. It's been being improved since then and it is basically the same things as we were talking about before, and almost identical functionality. And from a user's perspective, when they're looking at it, it looks just like Am Six Five looks just like for the most part what they're used to and it does all the same things except for it's a lot more cloud friendly. I guess I could talk. Go ahead. [00:15:15] Speaker B: One thing we probably should have mentioned is that as of the time of this cloud service release, adobe also announced that the current version of the product, Am Six Five, would be the last version of AEM that's released. They've released service packs and hot fixes to address security concerns and some minor feature upgrades. But for the most part, they've said there's not going to be a whole lot of development going on on the AEM Six Five version of the product. All of our engineering efforts going towards the cloud service. [00:15:48] Speaker A: That's right, yeah, we're coming right upon the Adobe Summit time, and usually right around Adobe Summit. Every year they've been releasing a new version all the way up until 2020. And so it was and then they just said, yes, we're not going to do this anymore. But the other thing too is that all of the maintenance agreements and so forth that you usually had to worry about in terms of I got to upgrade to the next version because they're deprecating the one that I'm on. Adobe did also say, okay, since Six Five is the last one, and since people are going to have notable challenges, there will be challenges. There are significant hurdles in some cases to being able to take an on premise or in network set of servers and move them to a cloud service. They said, okay, good, we'll give you all the time you need. Basically lots of runway. Lots of runway, exactly. Lots of runway. And that's going to be part of what we're going to talk about in the second half of this, too. What are those cases that still make it really difficult for you to migrate over to this? But I guess what I should talk about first is what problems does this solve? I guess above and beyond what a lot of times is just talked about in the marketing material. What are the things that this solves? One of those that I can just as a segue from the last slide is there is practical, automatic, out of the box without you having to do anything else about it. Auto scaling with this for the author, which is a huge deal, this is something that I've put all kinds like as a systems engineer, I put so much effort in trying to figure out ways around scaling issues, and I've made some of the largest computers I have ever worked on, just those authors. The largest computer I've ever worked on is one of these assets authors that is just like, I tried my best, there's no other way around. It just going to make the biggest fattest AWS instance that I can provision. But now there's auto scaling and it's out of the box. You don't have to do anything to configure it. You don't have to enable it. It's just a part of the product. And the same with on the user facing side. The publish facing side is automatic, out of the box auto scaling because of the fact there's some mumba jumbo behind the scenes that I don't think we're going to bore the audience with right now. But there was a separation of concerns between the factory code, the stuff that actually runs the product, and all of the content, which is alike, between all of the different the things that are mutable, things that can get updated while the application is running. Things like if the guy wants to publish a new version of the homepage or something like that, obviously the application shouldn't have to be restarted for that. But if there's new code or new factory code that needs to go out, then that gets stored in a separate location. [00:18:50] Speaker B: Now, one other thing that I think is important to note in the Am as a cloud service model and one piece that I'm still trying to fully implement for some of my customers who are stuck in the on premise world is that they separated out the concerns of working on the Web content management side and ingesting assets. So you talked earlier about that. People got to do a video shoot, and they have these massive video or they go out, do a photo shoot. One of my clients, they create about 100GB of new assets every day. [00:19:22] Speaker A: Right? [00:19:23] Speaker B: And for them, as you said, we've literally gone to the largest size of machine that AWS makes. And they're still like, why is this thing taking so long to ingest my photo shoot? Right? Whereas Adobe has said, you know what, within the cloud service version of the product, we're going to split those two things out from each other, there's really not necessarily a good reason to tie those two pieces together. And so they've created what they call the Assets microservice that allows you to more directly manipulate and work with the assets without impacting the people who are trying to create content and consume those assets. [00:19:57] Speaker A: Absolutely. And that was one of the first things that when the cloud service first was released and the website, the sites creation portion of it, still had a lot to be desired. The assets use case was immediately like, okay, why would I ever make this by myself ever again? Because, like you were saying, I had one of my customers that was a big Am Assets customer. We had to do a huge migration from other asset management systems into this. So it was like multiple terabytes that were going into this, and it was so fast to get them in. I was uploading it a lot of times from my desktop or in some cases from the customer's network. And I was getting wire speed on the way in. We're talking 500 megabit easy. It's just hauling in and then just renders it and it's through without any impact whatsoever to the people who are working in the authoring environment. Because it's going to these microservices that are just spinning up in Azure and then plopping the metadata and so forth into and one of the other things too with that is that Adobe has gone a lot further on those microservices with with other ways of of plugging additional functionality in there. Like as an example, you look at what what they've just with the Photoshop API. So I don't know if you've seen that recently, but yeah, it's just fabulous. You basically are taking all the power of lightroom and of the Photoshop desktop application or anything that you may have previously scripted with Photoshop Actions as a designer. Like if you get a layered file and you've got French, Italian, German, Spanish, Hungarian, whatever versions of a product and you need to flip it out, throw a drop shadow on it and then save out a PNG and a couple of JPEGs or something like that, that all can be done as part of an Am asset ingestion pipeline right now. It's a completely new world. All that stuff used to there was Mac Pros that were under desks that I used to see doing that kind of stuff and uploading them. But that's all part of the cloud service now. [00:22:05] Speaker B: Yeah, I'm excited to see where they go with that when they start to expand that beyond the Photoshop. For example, InDesign. I have a lot of customers that are heavy users of InDesign and are using AEM with InDesign server today to try and automate some of these workflows. And they're like chomping at the bit to get to something like that. Photoshop API, where they're just like, look, we just want to render 27 versions of this PDF in the main languages that we work with and all the content exists. It's all in different things in the InDesign file and within copy to manage the languages. Can't you just put those together for us? And the know Adobe Commerce, or Magento as what they used to call it, I think has been rebranded. I don't know, I'm not current with the stuff, but better integrations there, right. Like integrating AEM with the commerce platform form was always kind of a pain point. It was really difficult and hard to do and there were even vendors who sprung up in the market to try and make that stuff a little bit easier. And now Adobe just saying, you know what, we're just going to own this. We just need to make this something. [00:23:10] Speaker A: That'S part of the right, totally. Okay, so there's scalability. The other thing that you get with Adobe owning this now, because one of the more painful things about being a systems guy, being a project manager, being a product owner of an Aam site, one of the most difficult things was the fact that you were basically always in an upgrade cycle. It always seemed like only about a third of the time were you doing new development and like, oh, let's do a fluffy new front end of our site or something like that. We don't actually have an upgrade right now, but it seemed like the rest of the time you were in the middle of an upgrade cycle of, yeah, we're still doing a six two to six three upgrade or something like that. And this is one of the major goals of this was to make and this is Adobe's goal all along in this to make it so that upgrades were as painless as they could possibly be. And in this case, they're mostly automatic. And for the most part, they're doing so much testing behind the scenes and they've got customer like kind of canary instances and so forth that they've got actual customer code that they go and they take the new version. They run it on that customer code and they see if it's spitting out any errors and if it's okay before they release it into the wild for the rest of us. [00:24:30] Speaker B: Yeah, definitely is a very cool thing. And like you said, I saw that same pattern with many of my customers, where especially when Adobe was releasing a major version every summit that you'd have to go out well, I should say a minor version every summit, and you'd be like, okay, let's test all of our components against six three, and let's test them all against six four. And let's run the upgrade readiness tool that tells us if our components are using outdated APIs. But there is a little bit of a downside potentially to some people here. I've had multiple customers that are on the AEM as a cloud service that are feeling the pain of, well, Adobe just upgraded me and it seems I was using some API that either changed or is deprecated. And now I'm seeing impact on my production website because you mentioned that they do have customer canary instances that they're doing that. But that's largely the really big customers, right? If you're a little fish, you sometimes are just, you know what, it went live and deal with it. You better get your website fixed. [00:25:32] Speaker A: Yeah, it's true. Which I guess that segues well into talking about the pros and the cons of this because we've talked that there are some very obvious pros. The scalability is a huge pro. The fact that you don't have to worry about scalability is a huge pro. The fact that you're on a constantly once you have gotten yourself onto this platform, you're constantly on a vendor supported version of the platform. You don't have to worry about the version that you just purchased is going out of support in a year and you got to have to worry about spinning up a team to get you onto the next one. So that's nice. There are it overhead considerations which makes this attractive for a lot of teams, especially if you've got developer or a front end heavy team where you're mostly looking at new feature development and you don't necessarily have a big DevOps crew to support this. And if you can not have to pay a DevOps crew, which they're not cheap at all, and because of the fact that this is such an exclusive product with such a difficult runway to get in on, it's not like something that where there's a free version where people can get their hands on it. Like WordPress, where anybody with a website has used WordPress. Almost everybody in the It world has spun up a WordPress environment at some point in their lives, right. But whatever, only half a percent of them have spun up an Am environment or less. Right. So not having those DevOps guys that you have to pay is a big deal in some cases, or even if. [00:27:18] Speaker B: You have a DevOps team, but they're just never going to be the experts in AEM because it is so different from every other product that they're used to using. [00:27:26] Speaker A: Right? Yep, that's true. And then the other part of this is setting up a really repeatable and safe and high velocity continuous integration framework for the environment. And that is something where some of the largest companies that I have dealt with are still deploying manually by deploying to each individual server themselves and are still manually making changes to Apache and so forth. It's a historically hit and miss type of a thing. You have to have VPs who say, you know what? Nope, this is getting automated because the other ones just like, yeah, it costs a lot of money. Yeah, I guess we're not going to do that right now. And so the thing that you do get out of the cloud service is there's only one way to get code into the cloud service. It's Adobe's Cloud Manager, which is a web based set of APIs, but basically a front end that lets you go and get all your code in. There's only one ingest, only one ingestion mechanism to get the code in there and to get it tested and so forth. But you get that as a part of the product. You don't have to design your own CI CD framework. There is only one way. So if you have a particular set of requirements, that doesn't mean you don't have to do any of those things of designing your own CI CD around it. Because if you have particular requirements, you've got security scanning tools and things that you have to get in, you're still going to have to integrate that somehow, but you're not required to just to get basic automation. Like I want to deploy with the push of a button and I want to be able to roll back or something like that. Blue Green deployment is a part of this where it deploys the new live stuff while the current stuff is still being served. [00:29:22] Speaker B: And they do have, in addition to that CI CD pipeline, they have things like SonarCube. And some things like that, where they've set up some quality gates and things like that, that even a lot of the customers who have gotten to the automation pieces maybe don't have the sophistication level of AEM as a product that the Adobe engineering team does. To say, we can very clearly define some Sonar cube rules that identify patterns that we see thousands of customers misusing because we have so many different interactions with customers. And so I can write Sonar cube rules for myself, for my own implementation of AEM. But the broader the experience you have there, the more powerful something like that becomes. [00:30:07] Speaker A: All right, the cloud service does a bunch of awesome stuff. So there are, however, lots of companies that are still running am themselves, either via a vendor like yours and mine, or they're just doing it all in house. And why is that? Why are they still running? Is it just that they haven't gotten around to it? Or are there legitimate reasons still at this point because the product has gotten three years to mature at this point? Are there legitimate reasons that customers still have or things that the cloud service is still on the roadmap to be able to tackle use cases that it still has to be able to tackle? [00:30:53] Speaker B: Well, before we even get into the use cases, there are just some practical considerations that I hear from some of my customers that I think maybe ties into some of this. So, for example, I have one customer that did pay Adobe for a site license of AEM five years ago. And they've said, there's no way we're going away from we just dropped this huge chunk of money on what was supposed to be our platform for the next 20 years. And now they want us to turn around and look and pay them again to go to an annuity model, which is great for Adobe and it's great for the platform as a whole to have that constant revenue stream coming through. But this customer has said we've got to squeeze out the value that we can out of the existing cost or the existing expenditure that we did to get this far today. And on top of I may already have AEM cost. Then there's the what's the cost to do the engineering? How much work is it going to take me? As much as we all like to pretend like developers are perfect and they always do exactly what they're supposed to do all the time, there are a significant number of the pieces of AEM that in the cloud service model. You've been told this is off limits, this is taboo, you don't touch this anymore. This section of the application is read only now. And you have to go rearchitect the way you've maybe designed some of your solutions in the past to say, oh, it can work in that cloud model now, not to say it's ever impossible. I haven't seen a case yet where it was impossible to get there, but sometimes it's cost prohibitive to do that. [00:32:33] Speaker A: Yeah, I agree. [00:32:35] Speaker B: You and I have talked before about the data residency problems. One of the clients that I have is very cautious about ever putting any of their content out on the public website. They use Am strictly as an internal platform to manage the assets and the sites that consume those assets. That information is never released to a public website. It is for internal production use only. And while design teams can go in there and pull and they may publish a magazine that has some of those images in it, they don't have any reason or don't today see any reason for making that stuff available to other people. And so they're so cautious and they're so careful that they say, well, because this is only internal stuff, we don't want it out in the public cloud. We have it in our own AWS that we own, actually. In fact, that customer uses a lot of I can't remember what it's called now. It's one of the CNCF platforms where they basically are running AWS API compatible things on premise hardware. [00:33:42] Speaker A: Right, right. But it's physical gear. [00:33:46] Speaker B: They can walk down and reboot a server. [00:33:50] Speaker A: Yeah. So from a data residency standpoint there because that's a big deal and that's not everybody. Like you're an up and coming footwear company or something like that. Primarily you're media driven. You have to have excellent, like cutting edge commerce. You got lots and lots of media, video, stuff like that. Data residency is not and security is not necessarily on your top three or something like that. But one of the companies I did a lot of work for was a bank, big bank in Europe and data residency was a huge deal. And there's nothing that you're going to do. There's no dog and pony show that you could do, no silly dance that would convince that chief security officer any differently than, okay, if I say go get the hard drive that has this thing on it, you go get that hard drive, you know what I mean? Or something like this. When they released their quarterly reports, there was this whole dance around releasing quarterly reports whereby even like sysadmins, everybody was locked out of the system so that nobody could speculate on the data that was released in the quarterly report before the quarterly report was released. Having that get published through some workflow that would go through various clouds. Sure, you are taking the vendor's word that nobody could look at this on the way through, but could they look at it on the way through? Could somebody potentially speculate on this? And then if their answer is yes, then the CSI is going to say, sorry, that's no good. We're not going to do this. Additionally, all the data had to basically reside in EU. And that's one of the things that's very difficult because if you look at this diagram here, okay, so one of these things here, so you see this MongoDB portion of it, which is the Am author. So that MongoDB. I've got it on the diagram sitting at Adobe that MongoDB is actually MongoDB Atlas, which is Mongo's Host, the product. So it's not even necessarily 100% in Adobe control. Exactly. I mean, for the most part it is, but it is a separate product. So it's one of these things that could be a concern for somebody with very sensitive security requirements, like a bank or somebody where you're trading, which is the complete other end of data residency, which is China. China is its own thing. If anybody who's ever trying to do anything with China, that is a seriously unique dance that you have to do in order to get things to work in China. So I put together a little diagram here. This is I did for a blog post. But this just shows an example scenario is trying to show, like, okay, what could you do if you're trying to use Am as a cloud service, but you got to have step in China. Am as a cloud service doesn't work in China right now. So most of the services, like I said, there's MongoDB Atlas, but most of the other services are all Azure services. They're not all available in China, and many of them are available in China. So you can't necessarily deploy in mainland China right now. So you can deploy around mainland China. You can deploy near mainland China. You can deploy to Singapore, you can deploy to Tokyo, but you can't deploy inside of the Great Firewall. And there are problems like if you want to do commerce, for example, in China, your servers, your gear has to be physically in the borders. If you're going to do payment processing, it has to be physically there. The other things about if you're storing PII, it gets really gray as to whether or not it has to all of it has to be there. Does your author server have to be in China? It's also depending on what you've got on it, it's also gray. But it gets you into a set of really bizarre. So if you have a Chinese language site and you're selling to people in mainland China, like if you're like an auto brand or something like that, and you're using it because it seems like almost every auto brand is on AEM right now, your Teslas and Fords and GMs and so forth, they're all on AEM. But if you're going to go and run a brand, CN, it's got to be really tough to do that with Am as a cloud service. You're going to need to do something like in this diagram here, where you're publishing out and replicating to a set of Am publishers that are running Am 65 code, not the cloud service code. And you're running those inside China and those are hooking directly to your commerce gear and so forth that are in China and that you're Chinese. You've got a thing called an ICP license that you have to get from the Chinese government. All kinds of special regulatory stuff that you have to do that's all tied to that. Everything else there. But then you're as an It company, you're making two different code bases and maintaining two different code bases or having to have a code base that works on both, which is that's really not particularly easy. [00:38:56] Speaker B: Yeah. While you said China is kind of the 10,000 pound gorilla here, you gave that example of there are EU laws around some of this sometimes where it says, you know what, that information is not allowed to leave the European Union because it's got privacy information, it's got impact to EU citizens. So we can't have that hosted in the US. And things like, you know, there are many factors that you have to consider when you're saying, well, is AEM as a cloud service, right? For me. I'm on AEM on prem today. I need to look at going to a cloud service. And pretty much any partner with Adobe is going to be willing and able to help guide you through that decision making process. Our bread and butter is helping the customer get to the right decision. I shouldn't say making them make the right decision. We help them get to the right decision. [00:39:51] Speaker A: We have ways of making you talk. [00:39:53] Speaker B: Boy, if I could make customers make the right decision, something would go a lot easier. But because we've done this for so many various different implementations and different things, we can say. Well, what I know about your situation customer A I can take the information from Customer B through Z and I can apply. That and bring it back and say, here's what this is probably going to mean to you. Cost wise, team wise, decision making wise, and really help somebody get to that right? Does this make sense for you right now? [00:40:30] Speaker A: So cost and team, that's a really nuanced question also because if you take something like you take somebody who's running their own Am environment internally, right? So let's say they've got their own DevOps guys internally. They're running in a corporate data center. So a lot of those costs are kind of obfuscated to them. They're internalized costs that go into that, right? So maybe they're not out of their marketing budget. They're not paying huge amounts of money to a third party hosted service, right? So you've got that and you've already got the song cost on the salaries of the DevOps guys that you're paying right now. And you've got all kinds of internal systems that you still have to manage. You've got an internal elasticsearch cluster. You've got an internal whatever database, or you've got some other set of legacy websites or whatever the other gear is that you're running is still going to have to be run. So you just say, all right, well, we're moving our Am to the cloud service, so you're all fired. It's not going to work that way because there's still all this other gear that still has to be maintained. Sometimes those cost weighing the cost benefits there of like saying, hey, this license is a little bit more, but you don't have to pay to support this. For some companies, that's huge because that's the only gear. One of our customers was a brand new Am customer, brand new. And their Am didn't really talk to anything. It was just AEM. And so they're just a perfect one to move to the cloud service because there's nothing else really. It's like, oh, so when you move there, you don't have to pay a guy to run it, then good, we'll just redirect it there. And now it's all one thing. Now you just focus on developing on it. It's perfect. It was just right. But for some of these other ones, it's super complex. And like you said too, with the license that's already been acquired, if you just renewed your license, you've already bought it. And especially if your company is not a marketing company, like it's not a media company. Let's just say you've got like one of ours is they manufacture things. Like there's some parts, pieces, parts and stuff like that. Yes, they have to have all their they've got 20,000 SKUs or 50,000 SKUs or whatever they've got. So there's a lot of stuff to handle. So yes, Am is the right fit for that, but their marketing budget isn't gigantic. So when you say, okay, so it's going to be whatever, this many hundred thousand dollars to execute this move and this many hundred thousand dollars or half mil a year or something like that, that you go, I got that. So it's definitely a process to weigh it there. What have you seen in terms of internal, like back to the release process and the software development lifecycle of some of these big companies being able to whether or not they're able to meld that and move that to the new way in the cloud service. [00:43:26] Speaker B: Well, so a big piece of that is looking at our development pipeline and saying, okay, so if we're going to say we're going to go from on prem to the cloud service and it's going to take some amount I've not seen one yet where they just said I think you said you've seen one, but I certainly haven't seen one where they just said just take the existing code base and deploy it to the Cloud and everything just works. Right. There's always 90 days, isn't it? [00:43:55] Speaker A: 90 days, that's all you need. [00:43:57] Speaker B: We've definitely done the 90 day thing a few times, but that's like a tough 90 days. So one of the things you have to look at is you say you have to say, how much can I pause my existing pipeline? To say, I'm going to take all of those developers and shift their focus and say, no, I just need you guys take that 90 day window or whatever and get out onto the cloud service. Or maybe you don't want to pause the existing pipeline and you want to try and grow the team. Not something I would recommend because you're going to constantly be fighting like, oh, we're importing code. You guys are creating new things that we have to now migrate to the cloud service. But I certainly have had a customer ask me about that. They said, well, what if we just add more developers? I'm like, oh boy. Well, let's talk through what that means. And like you said, there is this kind of in my mind, I always want to go back to Ruby on Rails as the example. Know technologists out there who are listening to this or ever experienced Ruby on Rails. Ruby on Rails was an amazing product for building a website. As long as the only things you ever needed out of a website were things that fit on the Rails. [00:45:12] Speaker A: Right. [00:45:12] Speaker B: And that's why they called it on Rails. It's on rails. You don't get to change a whole lot. You're not supposed to be changing things. You're supposed to be doing things that are, hey, you know what? I did a blog in 15 minutes. That's great. I don't need a blog. I need an ecommerce website. Well, that's a whole different story. Right? [00:45:28] Speaker A: Right. [00:45:29] Speaker B: AEM is a cloud service, has a very similar feel to it where they've got this development pipeline, they've got these DevOps practices. And as long as you are 100% on board with the way Adobe is doing everything great, that's wonderful. But if you have any even slight differences of opinion on the way some of these things should be done, there's going to be a lot of organizational change. There's going to be a lot of just inertia, so to speak, of saying we've got to get in line there's. Adobe owns the GitHub repository where I push my code. Boy, I've had some customers that had some real heartache over that. And they're like, yeah, wait, Adobe has all of my source code out. Well, yeah, they do. That's going to be part of our deployment process is we deploy to an Adobe controlled git repository. There are customers that just, whoa, hold on. I'm not sure I'm on board with that. One of them was a credit card company that we talked to that was very concerned about that. Right. They're like, I'm not sure we want to do this. Another piece of it then is also just to say sometimes you have and I'm trying to I'll be honest, I'm more of an Ops guy than I am a developer. Have been Ops guy most of my career. I sometimes wear the developer hat and I can do so but every problem I look at is an Ops problem first. [00:46:51] Speaker A: Right. [00:46:51] Speaker B: And so I say that as a preface to say I'm not trying to be mean to developers, but certainly as an Ops guy, I sometimes look at decisions developers made and say, why on earth did you ever think that that was going to work at scale? [00:47:06] Speaker A: Right. [00:47:07] Speaker B: And as you look at going to the cloud, some of the things you have to look at is to say, are our developers skill set congruent with what's required to get things deployed? And working on the cloud, you may have people who again, I'm not trying to be offensive to anybody, but who let their skills stagnate a little bit, and they learned AEM five six, and they've maybe piecemealed together the differences between five six and six five, and they're able to get things deployed. But when you go to that cloud platform, it's a whole new ball of wax and they have to really sometimes buckle down and go back and resharpen that tool and figure out, oh, I've really got to rethink how I approach problems at a base level. [00:47:50] Speaker A: That's right. And a flip side of that same problem and coin is the fact that in deploying code into Aim as a cloud service, there are definitely constraints that Adobe has put on in terms of your code quality, in terms of your test coverage and so forth, to where they say if it's below this threshold or if it gets a poor grade on this set of tests that we're doing on it, it's actually not going up to production. And in some cases you have to do the I acknowledge my code is terrible to get it to dev, but even to get it to production they're saying, yeah, we're not going to allow you to override this. Because of the fact obviously that some of this is shared and they're allowing you to pay for the service based on your inbound traffic. So even if your code is awful and it ends up spinning up 55 publishers or something like that because it burns so much CPU, adobe is internalizing that cost for it. So they have to put some controls on that. Which means that in a lot of cases there's going to be more refactor than in some cases you thought you were going to have to do in order to get it to work on the cloud service. Now one other thing, just while we're on in Ops land. So one of the things, this is something that I just see gets overlooked so often and it's kind of my hobby horse that I ride sometimes, so I'll acknowledge that. But it's monitoring and log aggregation. And this is one thing that I think gets overlooked way too often. It's not so first off, log aggregation has never been part of the product that Adobe sells you. It's always you got to bring your own, which is great because there's a lot of great aggregators out there. There's a lot of great tools to be able to get that. And depending on what your costs and your requirements and so forth are, then you pick the one that works for your company. But this is kind of where the rubber meets the road. When you take a super complicated application server that is tied into everything that you have and has so many different ways that it can go wrong, so many different things that have to talk to each other and so forth, and you're going to move one of those pieces to a service that you don't control the gear for. You need to know what's happening because there's so many new ways for the experience that your users have to be impacted or the experience that your authors have to be impacted by software changes that you make that aren't necessarily visible anymore because you don't have your own data center going up in smoke. So to be able to have all those logs pulled in and so forth is a big deal. Yes, aim as a cloud service allows you to hook in your own Splunk instance and pull that in. It's fortunate because Splunk is my favorite log aggregator out there. So great. It's also the most expensive one, but okay, but it becomes a problem with some companies where like, I've been a few companies where they just said, you know what? So we realized that our monitoring was terrible. So we did a big shootout. We chose Dynatrace and so now we got Dynatrace everywhere. Good. Does Dynatrace work with a cloud service? No, it does not. [00:50:55] Speaker B: No. [00:50:59] Speaker A: And you can't deploy any Dynatrace gear on that because Dynatrace has this thing that it does where if it detects another APM on there, it will shut it down. New Relic is part of the cloud service product. And so yeah, you can't put Dynatrace on there. So, in other words, if your company, your entire enterprise, all the other gear that your company runs is running Dynatrace or Datadog or something else or App Dynamics or something like that, you're going to have a poor fit now with going to a service which you're using New Relic and you're using Splunk. And those are the tools. So being able to have one holistic your enterprise, your EVPE says, okay, give me a dashboard of everything you're like. It's going to be two. It's going to be two dashboards and you're going to have to pay a little bit more for this other product too. [00:51:47] Speaker B: I will caveat that just a little bit and say this is not something that Adobe loves for sure, but we have, in a couple instances managed to get customers elk stacks to pretend to be splunk for AEM and had to ship the splunk logs to the customer's elk stack and it just ingests into the customer's elk stack. It does take some work there's some configuration and there's a specific plugin you. [00:52:16] Speaker A: Have to like a log stash transform to be able to take the inbound Splunk hex stuff. Cool. [00:52:25] Speaker B: I can't remember what they call whatever the Splunk agent is. [00:52:31] Speaker A: Http endpoint connector. The HEC. Yeah. [00:52:34] Speaker B: Yes, there you yeah. So there is a log stash plugin that pretends to be the Splunk HTP endpoint connector and it will receive those messages from AEM. You tell Adobe it's not Splunk, but we're just asking you to just treat it like Splunk. [00:52:50] Speaker A: Please just ship it. [00:52:52] Speaker B: It's not your problem that it's not Splunk. [00:52:54] Speaker A: Right. [00:52:54] Speaker B: We will take care of that piece of it and then it gets into the customer's elk stack and they're able to do what they want with it. So that is one little like I said, Adobe doesn't love it. And there's always a long conversation with Adobe support when we do this, of Adobe saying, hey, we want to make this very clear that it's not our fault if something falls apart here. But yep, yep, we got it. We're accepting all the risk on behalf of the customer, but it's definitely a solution there that can sometimes be implemented. [00:53:20] Speaker A: That's a super lovely idea and now I'm actually going to have to go try that. I really love that. Okay, cool. So here's another one too. And we're only got like seven minutes until we're at an hour and I kind of want to see if we can somehow sneak us in there. So another one is Support structure. And this is one thing that is gosh, it's a deep subject, but there's one company that I did some consulting for and the entire reason why they couldn't be on Aim as a cloud service and the entire reason they weren't on Adobe Am managed Services, what came down to support and support is tough. It's tough to staff and it's tough because sometimes the guy you get is the entire face of the company and if he messes up, then you think that okay, all the support is bad there when it could have just been that guy's terrible day. But it is a serious part of the whole mix when you are on Aim as a cloud service. So because of the fact that it's shared gear, a lot of your ticketing is going into a shared queue. And unless you're paying quite a bit for your Premier Super Extra Platinum Grande. [00:54:38] Speaker B: Venti, they call it white glove support now, right? [00:54:41] Speaker A: White glove. Okay, good. All right, well I've had a couple of names that I can't keep up with that, but unless you're paying for that white glove support, then you're not necessarily going to get the SLA that you're used to. If you're a company which was used to, you've got James and Sandeep and they've got the pagers of Doom and that's it, you know what I mean? And I can call them up and they're on it in three minutes, sort of a thing. And if they're used to that sort of response time to being able to sort out an issue, then it's a little bit rough to go into a shared service based support model where you may not get the answer that you want unless you say five large, fire everything's down, sort of a thing. [00:55:27] Speaker B: Yeah. And like you said, sometimes what you get there is you may got the guy who was just hired by W yesterday and he hadn't heard of AEM a week ago. Right, right. And that's definitely a big part of the challenge. One of the things that we have done at Right Point as a company is to say on top of your Am, as a cloud service offering, we do offer what we call a run and operate contract where we basically will be the customer advocate. So if you have a problem, you call me. I have the pager of doom. I like that phrase by the way. I'd never heard that. I can't necessarily hop right on your problem and fix it. But I will be the guy that hounds Adobe over and over and over again. And I know people in Adobe and I can say, can you please just send this ticket over to Patrick? I know Patrick knows this problem and help you. He can help you figure it out. And so there are definitely partners that will help you in that space. But that is an added cost. Right. So you're already paying Adobe for one level of support. And I did have one customer who was very upset about this idea. They're like, wait, I'm paying Adobe for support and then I'm paying you to support me in. I'm like, well, you know that's. So we talked through what it was and we actually pulled up some tickets that they had filed with us. And I showed them. This is the amount of back and forth that it took me with Adobe to get your problem resolved. Now, do you really want someone at your organization having to have that deep AEM internals conversation with Adobe? I don't think you guys have the knowledge base for it. I don't think you guys have the time and I don't think you guys have the staff well, being able to. [00:57:05] Speaker A: Frame the problem so that a support engineer on Adobe side can successfully deal with it. Also, you've been around since before the War of 1812. So you know that in the old daycare days, you had to get the blessing of being able to get into the system and you had to basically prove your knowledge before they'd even let you in. [00:57:31] Speaker B: Yeah, you wanted to talk to Bernard or Dave. [00:57:33] Speaker A: Right. And so a lot of times that is the problem though is being able to have enough knowledge of how to frame the ticket so that you get the response that you need. Because sometimes if you're a customer and all you. Know is that the website's app or some pages are showing an error and you don't know enough of like, okay, I think I know what this is. So now this is the actual problem and then you know what you need to have them take internally to engineering or whatever. [00:58:11] Speaker B: It's the difference between, I don't know, some of my pages don't render and the saying every page with this specific component on it is erroring out. Right? [00:58:20] Speaker A: That's right. [00:58:20] Speaker B: A lot of times the customer is not going to know how to even get to that piece of information. [00:58:25] Speaker A: Right. Or if there's one of these things where as an experienced engineer, you are pretty extra certain it was the service pack that broke it and you're pretty extra certain that there is going to need to be a factory fix that comes from that for this. Right. And you're able to prove that with prove your hunch on that to the person and write. Whereas a lot of times, if you don't have enough info, you're not going to be able to convince the guy on the other end or it's going to take you a month to convince the guy on the other end that it was the service pack that broke. It not just the customer is being silly with their code or something like that. [00:59:04] Speaker B: Yeah, it definitely helps when I can point to like a sling commit and I can say, look this right here's. [00:59:12] Speaker A: No, no, our arbor does the same thing in terms of layering support onto Aim as a cloud service because it's unfortunate, but it's a needed part of the Adobe ecosystem at this point to be able to especially if you've got needs like that. Because not every crew that runs am is the same. And in some cases you need people that have been running it for a while because in some cases too, especially. [00:59:42] Speaker B: I think as the cloud service matures, we'll see more and more people that are just saying, you know what, it's no longer worth it for me to keep this skill set up. Adobe is going to handle 99% of the cases. The other 5% I can find an arbory or a right point that will take care of that for me so that those skill sets will atrophy. People will start forgetting how AEM works internally and they'll be needing more and more of that. Like you said, that ecosystem support. [01:00:11] Speaker A: That's right. I mean, additionally, I've had plenty of customers where a lot of times my point of contact is now working at. So a lot of those guys who were the good guys were like, oh yeah, I know that. I talk at this customer, he's not there anymore, he's at still. This is going to be a necessary part of the ecosystem, I think, for soundtime. All right, well, okay, good. So now here we are at an hour. So I don't know that we talked about or really definitively answered the question of is it still a thing for everybody? But I think that we hit it on enough points about whether or not it's still a thing for some people. [01:00:55] Speaker B: And some of the questions you'd have to ask yourself right. To say, are we making the right decision staying on premises, or are we somebody who should maybe start evaluating that cloud? [01:01:05] Speaker A: Indeed. Yes, indeed. All right, well, this has been great, Joey, and yeah, I'm glad we did this and hope we get to do another one. [01:01:15] Speaker B: Yeah, for sure. Some of the things that we didn't cover that you had kind of laid out in the initial outline, I'd love to get back together and maybe circle back on some of those. And also, as we talked about previously, WritePoint does have an offering called Cord Tools that is similar to AEM as a cloud service. We've done some of the same things that Adobe has done, but we help you implement those on your own hardware or in your own cloud account. [01:01:45] Speaker A: I absolutely want to just do one. [01:01:48] Speaker B: I hope you don't mind me throwing the plugin. [01:01:50] Speaker A: No, I absolutely don't, because I think that that is something that deserves an episode, because I've actually talked about the work that you guys have put into that with a few of my people. They're like, wow, they didn't a lot of work there. [01:02:04] Speaker B: There's parts of, actually, of AEM as a cloud service where I've talked to Justin Edelson. I'm like, you stole my work there. Justin's like, well, you know, you and I talked about this years. [01:02:20] Speaker A: Oh, my goodness gracious. All right, good. Yeah, let's definitely do this. Definitely do this future episode planning. All right, good. Well, great talking to you, Joey. And until next time. [01:02:30] Speaker B: Thanks, dad. [01:02:31] Speaker A: All right, don't.