It’s a great concept: “some of the biggest stores” offering up a unified experience intended to streamline the shopping experience — and in the meantime hitting Amazon where it hurts. But in practice, ShopRunner, still in beta, has a long way to go before it matches Amazon’s general ease of use.

To see for myself, I signed up for ShopRunner’s convenient 30-day trial. (At the end of the trial you pay the annual $79 fee, just like Amazon Prime.) After digging around for several hours for everything from DVDs to, say, flowers, here are my observations:

Slim pickings

While more than 40 stores have agreed to participate with ShopRunner by year’s end, only 15 stores are currently active. This means the offerings are pretty limited right now. Case in point: My search for the recently released Iron Man 2 on DVD yielded kitschy Halloween costumes for women. Even when all 40-plus stores are aboard, you have to wonder if they will be able to compete with Amazon, which has hundreds of thousands of big and independent vendors selling new and used products.

Beta? Try alpha

While its interface seems welcoming enough, ShopRunner isn’t kidding when it warns that it’s still in beta. Six of the 11 main product categories take you to a “Coming Soon” page. So unless you’re looking for kids’ toys or, say, beauty products, you’re out of luck. While many companies launch in beta, it’s rare to see something so skeletal — especially from major brands. So far, you can’t access more than 50% of store offerings. So what’s the point in stopping by? For now, it’s far easier to troll the different retailers’ sites independently.

Multi-multi-click instead of one-click

Part of what makes Amazon (and Amazon Prime) convenient is the “one-click” option: Sign in once, place items in your cart, and then you’re ready to order. The ShopRunner shopping experience doesn’t come close — at least not yet. Even if you sign in to ShopRunner and use the aggregated Marketplace search engine, you’re redirected via another window or tab to the actual vendor site — where you then need to register and/or sign in again to finalize the purchase.

No universal gift cards

One feature Amazon has really nailed is the gift card. Whether you get a virtual gift card or a tangible one, you can input the code and apply it to your account, and Amazon automatically deducts the amount from future purchases. With ShopRunner, that option doesn’t exist, so you’ll need to stick with gift cards from the various participating vendors.

Limited exclusivity

Some of the retailers who have signed up to participate in ShopRunner are still selling their goods through Amazon Prime, including the NFL. “We believe we could take an exclusive approach, but this wouldn’t be in the best interest of members,” said Mike Golden, president of ShopRunner, to the Wall Street Journal. That’s one way to look at it. But given the short list of participating stores, it doesn’t help that some of them are crossing the line into “enemy territory.” Why even bother with ShopRunner if users can get some of the exact same items from the exact same vendors via Amazon? The nonexclusive practices from vendors surely won’t help ShopRunner differentiate itself.

Now, before some accuse me of not giving ShopRunner a chance, I actually tried purchasing an item from Toys “R” Us this morning. I logged in to my ShopRunner account on ShopRunner.com, randomly searched out a Nickelodeon product via Marketplace, and tried to purchase it. Not only was I shuttled to the Toys “R” Us site, but I received a confusing prompt requesting me to purchase another $92.02 (or so) of items for me to qualify for free shipping. Nowhere was there a ShopRunner log-in as I waded through five pages of account registration.

“Any time you have a baby program going against something that’s four year’s old is sort of an unfair comparison,” says Fiona Dias, Executive Vice President of Strategy and Marketing at GSI Commerce. ShopRunner will begin advertising to consumers in November, says Dias, to allow time for the site’s kinks to be worked out between now and then. By the end of the year, users can expect 40 participating retailers offering over 5 million items to choose from.

“By that time, it’ll be a very broad assortment,” assures Dias.

While ShopRunner has a compelling idea and a strong roster of vendors, it doesn’t show any sign of being able to compete with Amazon’s offerings. If anything, the service is a reminder to e-commerce consumers of reasons to keep using Prime: ease of use, countless large merchants and independent vendors to choose from, plus a wide range of prices on new and used goods. If the team behind ShopRunner is smart — and the folks at GSI certainly are — they’ll improve the interface and user experience pronto before Amazon catches on and does what it does best: undercut prices.