Earlier this year, Microsoft Gaming CEO Phil Spencer outlined his vision of “a future where every screen is an Xbox.” This idea relies heavily on Xbox Cloud Gaming, which lets you stream games to a phone, tablet, TV screen, VR headset, or laptop, with all the heavy computing handled by Microsoft’s cloud computers. Yet this month, Microsoft released a mild refresh of its Xbox consoles. Is there a place for them in Microsoft’s vision?

That’s what I wanted to find out while testing the newest Xbox Series X. In the ideal world, playing on a console wouldn’t be any different from playing in the cloud. But this is what separates “every screen is an Xbox” from “some screens are kind of an Xbox, sometimes.” So, let’s start there.

The Latency Problem

Xbox Cloud Gaming still hasn’t shed its “beta” branding, and for good reason—but not the one you might think. Streaming games from the cloud has quietly become extremely doable. With a fast-enough internet connection, streaming 60 frames of HD (or even 4K) video to your laptop or phone can be a trivial task. According to data from the Federal Communications Commission, most of the United States has access to at least a 250 Mbps connection, while around 47 percent of homes have access to gigabit connections. In other words, it’s fast enough.

The problem isn’t bandwidth though, it’s latency. Video games are built on instant feedback. Some games can fudge the definition of “instant” a bit, but anything more than half a second of delay between input and action would be unplayable for even the most forgiving games.

This is the problem I faced when trying out Xbox Cloud Gaming on an Amazon Fire TV Stick 4K Max. On paper, this is one of the most accessible ways to get into Xbox gaming. With this $60 streaming stick and a Bluetooth Xbox controller, you can stream games from the cloud to any device that has an HDMI input.

In practice, the input lag was problematic. I tried playing Starfield on the Fire TV, and I was able to role-play just fine, but as soon as combat began, I felt like I was learning how to control a marionette in real time. Under gunfire. I’d briefly flick my control stick in a direction for a moment, let go, and watch the stick snap back to its default position, and then my character would move. That’s not a good way to dodge bullets. It worked a bit better for more casual games like Donut County, but even then it felt a bit like dictating an email to someone who’s a slow typer.

Bluetooth is fine for basic things like streaming music, but when you need to sync visuals and inputs to audio, the cracks start showing. Add that to the natural latency that comes from pinging a remote server somewhere in the world, and it can feel pretty bad. Even when playing locally, console manufacturers either use proprietary wireless protocols or add custom enhancements on top of Bluetooth to deal with the latency problem.

One fix is to use a controller that skips Bluetooth and connects directly to Wi-Fi. This is something Google’s ill-fated Stadia tried—and Microsoft may also be working on—which takes out one link in the chain. Currently, my Xbox controller connects via Bluetooth to my Fire TV Stick, which then passes that signal onto my Wi-Fi router, each step adding precious milliseconds.

A Wi-Fi controller could help cut that down. But until Wi-Fi—or some version of low-latency—controllers are common, cloud streaming is always going to struggle. It might still struggle even with such a device, as long as internet speeds vary so widely by region. This means there will probably still be a need for a console to play games locally. But does it have to be an Xbox?

The Windows Factor

Microsoft may own Windows, but Valve has dominated the PC gaming space for more than a decade. Valve operates the Steam gaming storefront, and while its dominance is slowly (very slowly) eroding, it’s almost a guarantee that you have a Steam library with hundreds of games if you play on a desktop or laptop.

There’s little reason for PC players to buy games via the Xbox Store instead of Steam (or Epic or GOG), but Xbox Game Pass changes that equation. For $10 to $20 a month, players can get access to a generous library of games. It’s a compelling deal that’s brought in over 34 million subscribers, which may even be more than the total number of Xbox Series X/S units sold.

The experience of Game Pass on PC is delightful. While writing this story, I discovered that We Love Katamari REROLL was recently added to Game Pass. I decided to replay Donut County, and my partner has spent countless hours in Coral Island. None of which we’ve spent an extra dime on.

In a weird quirk, Xbox Cloud Gaming works on a PC, but only if you have a connected controller. This was a mild annoyance because, while I prefer to play Starfield on my Xbox Series X, I would’ve liked the ability to hop in briefly on PC to check on my outposts. I can install Starfield locally and my game saves will sync over, but the option to use the cloud installation from my laptop would’ve been nice.

Even still, I can see glimpses of the future Microsoft is envisioning here. When I first bought Starfield via Steam, I accepted the limitations that choice put on me. I could play easily on my desktop, but if I wanted to play on the couch, I would need to connect a computer to my TV. At the time, I only had an Xbox Series S and wanted better performance than that console could offer. With the Series X, it’s easy to play in the living room, and then pick up the game on my PC later.

Gaming handhelds are a whole other issue, and I have plenty of thoughts about the problems with Windows handhelds. They’re clunky, the interface isn’t very well optimized, and Windows adds a ton of overhead that wastes processing power and battery life that gaming handhelds desperately need. That’s one area where Valve has an advantage with the Steam Deck. But it’s getting easier to imagine a future where I can buy a game (or Game Pass subscription) once and play it in my living room, at my desk, or on an airplane—all on one platform that syncs my progress. We’re not there yet, but I can see the pieces.

The Console Present

So where does the Xbox fit in? I tested out the new refresh to the Xbox Series X, and it’s hardly different from the original. If you own the original Series X, you do not need to upgrade. Full stop. Calling it an upgrade is a stretch. Sure, you get more internal storage, but you lose the disc drive. You’re better off upgrading with an expansion card. The refresh simply hasn’t changed the math on whether the console is worth it in the same way that the PS5 Pro might.

But that doesn’t mean the Xbox isn’t worth it for everyone else. I played a lot of Starfield with a desktop computer hooked up to my TV and a Bluetooth controller. While it was mostly fine, I had to sit on just the right part of my couch to avoid accidental disconnects. I got more frames and better image quality than I would have on my Series S, but now I’m wondering if I would’ve been happier just playing on a Series X instead.

The appeal of the Xbox ecosystem is growing as Microsoft adds (read: buys) a ton of major games to its library, but the Xbox console itself remains less appealing simply because it’s only one of many ways to play. You need a PlayStation 5 to play games like Final Fantasy VII: Rebirth or The Last of Us Part II. You don’t need an Xbox to play Starfield.

But it doesn’t matter whether an Xbox is the only way you can play a particular game. What matters is whether it works for how you want to play. And as long as people like playing big games on the couch with minimal hassle, the Xbox is probably going to keep having its place. A proper hardware upgrade sometime soon probably wouldn’t go unappreciated though.