[Transcript] Bing Ads Advertiser Science Series – The Science of Remarketing

Bing Ads Advertiser Science Series latest discussion topic was about Remarketing and the nuances that can make businesses benefit from it.  Discussing the remarketing tactics in detail, Frances Donegan-Ryan (SEM Pro Marketing Manager, Bing Ads), Purna Virji (Sr. Bing Ads Client Trainer, Bing Ads), Christopher Walleck (Program Manager, Bing Ads) shared some insightful tips and tricks about getting started with remarketing in Bing Ads.

Moderated by MJ DePalma, the session also had Ben Carson, Global SEM Lead, Microsoft Store, Microsoft and Natalie Barreda, Senior Client Manager, Point It Agency throwing some light on how Microsoft optimized for a return on ad spend

You can view the entire session here.

For reference, here’s the transcript of the video.

MJ: Welcome everyone, to the Bing Ads webcast on the Science of Remarketing, next level optimization techniques for remarketing with Bing Ads. My name is MJ DePalma, and I will be your host today. This is the fourth webcast with our Advertiser Science series, and we are excited to share strategies and messages from our own Bing Ads experts as well as the perspective and direct experience from one of our customers and our agency.

Before we begin, let’s review some housekeeping items to enhance your experience with us today. Our webcast console that you see is customizable on your side, where you can click the content widgets that you see on your screen and move them around or size them for your screen size. You can expand your slide area by clicking on the ‘Maximize’ icon on the top right of the slide area, or by dragging the bottom right corner of the slide area. We recommend, if you can, to leave the widgets as they are, so you have the ability to see all the great resources at the same time.

For Q & A, please feel free to input your questions as you have them and we’ll try to answer them as we go. But for the most part, we’ll get to the bulk of the questions at the end of the webcast and try to answer as many as possible. If we run out of time in answering all incoming questions, they will be answered via a blog post. We do capture all questions. You’ll see a ‘Resource’ widget in the upper right hand that we’ll refer to during the webcast.

You can go ahead now, and download the‘t-test’ Excel Spreadsheet, so you’re ready to follow along later in the presentation, and also for your convenience, the deck is already available for download there as well. Also there are awesome links to stay in touch with us in the deck, and there’s also a Twitter widget that will enable you to tweet right from the webinar with the Bing Ads hash tag, #BIngAdsWebinars, which appears to you upper right under resources. We will also provide all twitter handles of the speakers in the next slide.

Also the On-demand version of the webcast will be available approximately four days after this webcast, and it can be accessed at the same link that you registered with. If your colleagues or friends could not make this time, not to worry; they can still click the ‘Invite’ link, and they will be taken to the Registration Page for the On-demand version.

Last housekeeping item, we would love your participation in our survey at the end of the webcast, so that we can improve in any way possible.

So, let’s begin by introducing who we have on the webcast today. Frances Donegan-Ryan’s here; she’s the SEM Pro Marketing Manager. Also we have Purna Virji, one of our Client Trainers on the Bing Ads Academy Team. And we have Christopher Walleck who’s the Program Manager who specializes in remarketing. And towards the end of the webcast, we have our special guest speakers here today – one of our customers who happens to also works for Microsoft is actually the SEM Global Lead, Ben Carson, and he is responsible for all of the SEM programs for microsoftstore.com, as well as his Agency Contact and Lead, Natalie Barreda. Super excited to have them here.

Now let’s review the agenda, and what today’s webcast is going to go over. And it’s really in three sections. So the first section we’re going to go over, we’ll cover ‘Strategies’ and the ‘Set-Up’ for remarketing, and Frances and Purna will really dig in deep there. And then we’ll have Chris Walleck who’s going to get into the ‘Next Level Optimization Techniques’ and really describe how to run some ad experiments as they relate to remarketing. And then, last but definitely not the least, we’ll hear from one of our Bing Ads customers, as I mentioned, Ben and his Agency Lead, Natalie here, who is in the room with us. So without further ado, I want to pass the mike over to Purna. Purna, the floor is yours.

Purna Virji, Sr. Bing Ads Client Trainer, Bing Ads, Microsoft:

Purna: Thank you MJ. So, I love this quote from our CEO, Satya Nadella: “The true scarce commodity is increasingly human attention.” That’s so true. Here’s [e-phone 04:26] fact: You’re going to check your phone or your e-mail or your social feeds at least 20 times before the end of this one hour webinar, that are constantly bombarded by data. Just take a look at your screen like that. There is new devices, new apps, new services every day, each is holding the promise of helping us get something more done with less effort, but it’s just one more thing competing for our finite time and attention. So, how do we as marketers cut through this clutter? That’s what Frances and I are going to talk through today.

Frances Donegan-Ryan, SEM Pro Marketing Manager, Bing Ads, Mircrosoft:

Frances: Thanks Purna. Hi everyone. This is Frances, and I’m thrilled to be chatting with you today. I work on the marketing team here for Bing Ads and something we always talk about with our own marketing efforts, and I know that it’s something customers talk about, is the need for marketing, whether you’re doing it in person, in a store, or online, or over social media, is that it has to resemble one-on-one conversation and we don’t always have the luxury of being in person with our customers or seeing them face to face, but even our work online really needs to feel that way if they’re going to trust us, and they’re going to invest in us.

And so, talking about customers who are online, if we’re looking at searchers, those potential customers, there are lot of different types of people online who are searching for you. So some of it might be brand familiarity. They know your brand, they know your product, so they previously visited your site. And they’re valuable, because they’re going to recognize your name or your product name, and click on your ad, and that click is probably going to be very high quality because there’s a greater chance that they’re going to convert and purchase your product.

But what we see, and what I’m sure most of you have seen in your analytics, whether from your analytics or other analytics that you’re using, is that 96% of first time visitors leave a website without converting. So a huge amount of the people who are going to come and visit your store, visit your services, visit your web page are going to jump back off of that page, and only 70% of shoppers who put products in their shopping cart… Sorry, only 30% actually convert, about 70% of shoppers are what we call ‘abandoning shoppers’, so they’re adding things to their cart , and then they’re walking away and they’re not purchasing.

Purna: Huge.

Frances: Yeah, that’s a huge number. And those 70%, just because they didn’t convert, I think in that time, Purna, you can agree, they’re still super valuable customers.

Purna: Oh, absolutely.

Frances: So what we’re going to chat about today is this second chance. How do we reach out and use paid search and use remarketing to get a second chance with those customers? So Purna, I’m going to hand it back to you so you can let people know how to get started in doing this.

Purna: Sounds great. Thank you, Frances. So yes, this second chance, we need to get them back by working on our Bing Ads Remarketing product. So there are essentially five steps to getting started. If you’ve ever worked with Google Ad Words RLSA, it’s a very similar product offering that we have with some, of course, differences. So let me walk you through how do you get started with our Bing Ads remarketing. Step 1: You tag your website with our UET tag. Step 2: You want to define your audiences. Step 3: You’re going to add list to Ad Group, then you’ll have fun with bids and creatives.

So let me jump through to give you a one-second overview of our UET product. As you guys know that UET tag is our Java Script tag that you need to place across every page on your website in order to not just track conversion goal but also how people are interacting with different parts of your website. We do ask that you put it on every single page of your site simply for the visibility that we can get for remarketing. We can create far more effective lists if we can get a better idea of how people are interacting and engaging with the different sections of your website.

Okay, so I was talking about remarketing a little bit. So how does all of this work? Similar to RLSA on Google AdWords which you’re familiar with. The people, the user will come to your website and then they get added to one of the… they get cookied and then fall into one of the remarketing lists. The user will then leave, but if they go back on Bing and do another search on the Bing network, then we can serve them an ad, either a custom ad for a custom keyword, or your same ad. So it’s very simple. It’s really a good way to speak to them and serve a bit more personalized and customized content to them.

Let me walk you through this set-up. The step 1 is you want to go to “Shared Library,” which will be on the left side of your Wonder Bar, and then you want to select “View Audiences” under “Audiences.” Next, you want to hit on, once you go into the “Audiences” tab, you want to select the button that says “Create Remarketing List.” It’s right up front, hard to miss. And then, you go into creating your actual remarketing list.

So you set it up, you give it a name, and then you define your rows, so you could have people who visited a certain page, then you can add rows and get more interesting, “They visited this page, and they’ve visited this page, so I’m going to put them in this list” or , They visited this page but they didn’t visit this page, then that’s fine” or “They spent time on the site” or any of the other goals that our UET tag offered.

Then you set your remarketing pixel length. You define the length. We can go to a max. of 180 days, and you select your tag and the scope, and you’re set. Super simple with creating the list. Then you need to associate it with your ad group. So we’re going to go into “Audiences,” and then select “Create Association,” and from there next we’re going to select the ad group that we want; so we select the name of the ad group, and then all we do is, add the remarketing list to the ad group. Fairly simple, right? You’ve done this before, it should be no problem.

Couple of things just to add: We can also do bid mods., so if you wanted to exclude certain audiences, say, people who’ve converted before, you can do a decreased bid, you can bid mod. down to (-90%) to lower your chances of them seeing your audience, and we can do bid mod. all the way up to (+900%) to increase the probability of people wanting to buy or being seen. If you’re familiar with the “Bid Only” as well as “Target and Bids” we also have that option, right Frances?

Frances: Yeah, and so I think, just so that people know, we’ll talk more about what are some of the guidelines to use when you are setting your bid adjustments, but this is a really powerful tool to use because you might use different bid adjustments with different types of lists of people. And I guess the most important thing to know how to really use bid adjustments effectively is you need to know what the value of your customer is and what type of customer. So if you understand the lifetime value of your customer, a customer that you have in different segments, if they’re buying a high-end product or a medium product, or a smaller product, once you know that information, then you can get super sophisticated with these bid adjustments.

Purna: Sounds absolutely correct, Frances.

Frances: So then the “Audience Bid Options’, Purna, you want to go over what you can do here?

Purna: Absolutely. Well, (+900%) and then (-90%) is great for bid mods., and I just wanted jump, to ensure we talk about their latest “Target and Bid” or “Bid Only”, and then Frances, why don’t you explain a little bit more about when you would do “Target and Bid” versus “Bid Only?”

Frances: Yeah, so if your strategy is just adjusting bids, so, you know, you have one list and you just have left that list alone, and you’re not doing any other customization, that’s when you just want to select your “Bid Only” setting. But if you’re customizing ads, so say, for example, you have a remarketing list that’s going back to those shoppers, so those 70% of shoppers who put something in the shopping cart but then left before purchasing, you would probably create a list of those people, those “Abandon Carters” and if you want to add a custom ad, so I’m sure this…I just got an e-mail like this this morning, “I put something in a cart, I didn’t go through and complete it and that was about two days ago.

And this morning I got an e-mail from the brand saying, ‘Hey, we saw you left it in your shopping cart. Here’s $10 off your order, or if you have any questions, e-mail us.’” So you probably are already doing that with your e-mail marketing strategy, you can also essentially do the same thing with your paid search strategy. So they know that you’ve added something in the cart, they know it’s pretty recent, right? It was in the last week or the last five days. They can retarget me with an ad, giving me a discount code to encourage me to really go and complete that purchase.

So if you’re customizing ads, then that’s when you want to set a target and a bid. So you want to say, “I only want to target these people from this marketing list and I want it if that person is….” “If Frances searches again for this product, this keyword, I want to up my bid so that I know my ad is shown, you know, in the first position in main line, and I want to also show her coupon code in that ad.”

That’s when you’re both targeting and increasing your bid, because you’re happy that paying a little bit more for me to do, really think that at that stage I’m going to convert. Same thing, so you can customize your ads and then you can also broaden your keywords, and Purna will get into a little bit more detail about broadening keywords, but if you’re going to expand the number of keywords that this ad is going to show, then you also want to “Target and Bid” there as well.

Purna: Exactly. I’m super excited to go into some of the strategies that we’ve put together because as we see, all this is wonderful to set it up, but ultimately I want to, on my promotion, I want to get the kudos at the office, make my client happy. So how do we use remarketing to make money? So I want to walk you through five test score list that we can use very cleverly. So let me jump to the first one. That is what we call the ‘window shopper’.

So who gets on the list? These are the guys that, they sort of they came, they had a very surface-level interaction with your site, but we couldn’t really do anything much with them. They didn’t express too much more interest, may they went to like a Home Page or like a broader higher level page, not a deeper category, they looked around but then, may be they just left.

We don’t know what happened. We want to bring them back. They’ve come to our site once; as Frances was saying earlier, if they’re more familiar with you, there’s a higher likelihood of converting. So they aren’t familiar, so let’s bring them back. Here’s what we want to do, is broaden our keywords, because these keywords here, it might be very unaffordable sometimes to bid on the really high quality or the really broad keyword, but if you’re doing it with remarketing, it’s less irrelevant, and it would be a good way to bid on broader keywords.

Like “Shoes”, if I was a shoe store, would be too unaffordable before, but now with remarketing, because it’s only the bids for the people who’ve been to our site before, it’s affordable. So I would do a increase of bid mod., may be give them some Ad Copy to incentivize them to want to come back. So, things here could be brand and product reminders for what we call TOMA or ‘Top Of Mind Arena’.

Let’s go to the next list. These are the “Abandoners.” So these are the people that came all the way to the cart page, like, those 70% of people, right? Or like Frances herself, “We reached the cart page, but then we left. May be we got distracted by the latest cat video?” I don’t know. Or it’s take a selfie; may be the lighting was so good, but if they were very engaged, these people, they wanted to buy the product, they were so close but they didn’t go through. How do we get them back?

Frances: For me, it’s really that… Purna, I was going to say for me it’s usually that I have to go and find my wallet somewhere to get my credit card number and then I get distracted, and then I’m probably also late to get somewhere, and there is a lot of things that happen.

Purna: Oh yes, that’s me as well. I’m super ADD, like I’ll go and look at something, and then I’m like, ‘Oh!” distracted by this stock, and distracted by this stock, so yeah, I certainly feel that. So anyways, but to get people like us back, the advertiser can do is like, you know, may be give them a little bit more information. May be they want to be encouraged, like, “Hey, don’t worry, we process fine. We list return policy.” Or maybe, you know, you don’t always want to give away something for free, because you don’t want to sort of cheapen your value, but you could say that, “Hey, just this weekend, I’ll upgrade you to like one day shipping.”

So it’s something that just a bit of incentive and people who want to come. Also my favorite, really sneaky use of RLSA, especially with abandoner, is to bid on competitors terms because people compare them and shop all the time. I do it; Frances, you do it; MJ, you do it; like we all do it. If I’m bidding on my competitor’s term, then I just make sure that my remarketing ads have the more compelling offer – that’s really more exciting.

But you would also want to set your cookie windows a little bit shorter for this one because it’s more immediate – the chances of them converting is pretty much more immediate. Then you can also consider raising your bids as high as you want to, because this is the audience that’s most likely to buy. And again, special discounts always work, right? This is really great stuff to do.

Frances: Yeah. Purna, I just want to reiterate that point you made that you can bid on your competitor’s keywords. We get this question all the time, and people like, “Oh my gosh, I’m allowed to do that?” And yes, you’re allowed to bid on competitors’ keywords, competitor branding. You’re not allowed to use their branding in your ad copy, or in your ad title. So don’t think that you can’t compete in that way; you just can’t use it in your ad copy.

Purna: Exactly. Now those are excellent distinctions, I’m so glad you made that point, Frances. Yes, it’s a constant question, so bid on the terms, but you just have to talk about yourself and your features. Alright, let’s go the next list. So, this is to re-engage the engaged visitors. So I call this… I compare this list to… You go to the bar, you talk to somebody, and you have a great conversation, like three hours, and it’s fabulous. You give them your phone number, but they never call you. It’s like, “Gosh, they were so interested, but they never converted.”

So this list, I would say, is in between the abandoners and in between the window shoppers, because they showed a little bit more interest, but they never got as far as calling you. It’s so sad. Well may be what we can do with them is you could define this list as, may be somebody who’s watched part of our video, so custom events-based…or may be the number of pages or the types of pages that they visited, any of these could be a good list.

So they’re interested, so how do I get them to pick up the phone or actually do it further down the conversion funnel? Let me look at some of the things that we can do. Perhaps they need a little bit more information about you, so do you think you could use your ad extensions really [Inaudible 0:21:20.6] here, so that’s a good one.

May be link to product reviews, some case studies, do them in your cyclic extensions, or I know that we have some exciting review extensions that have been announced, that are in really pilot, so hopefully stay tuned. May be give them a special shipping offer, or “Buy this, and you’ll also….we’ll throw in this for you as well.”

So this list we found that, may be like, a 150% bid mod. is good, and also consider sending them to the phone. The phone calls tend to convert like, at least 3 to 10% higher than the actual website, so if you try to get them to call you instead, you could probably convince them on the phone to convert for you. So that’s a great other tip to use for this list.

Frances: Purna, I just want to jump in there quickly as we’ve got some questions with different people asking about how remarketing works with things like a car dealership, where it’s not an online purchase or other big ticket items, and I think the tactics that you just talked about are what you want to use there. So if you’re trying to get someone to call, using “Click to Call,” if you know that you’re going to convert over the phone, or perhaps you don’t sell things online, but you’re taking leads over the phone or collecting customer information that way, use the “Click to Call” ad extension.

Purna: Yes, or location extensions, so just tell them how convenient it is to the location and then you can use these remarketing ads to focus on your future, like may be you’re like, “Hey, for this call, I’d look at why we have such a fantastic safety rating.” Things that you can do to convince people that this is the right product or the right cost for them, so you can do this fabulous use of remarketing, because you can think higher funnel, terms like, if I was looking for a car for when my son starts to drive, it would be like, “What’s the safest car for teenager?”. And then I would love to see an ad that showed up there talking about how safe my car is, so that will be something wonderful to share.

[0:23:22.2 crosstalk]

MJ: This is MJ. I actually also worked in the past with some car dealerships, and one of the tactics that worked really well was, in each of their funnel they had, you know, “Request a test drive.” Looking at a brochure and just tagging those different pieces of the funnel could really move someone into finally that “Click to Call” to actually schedule. So think about, like, your customer’s journey, and what those important action items that indicates that they’re really interested. And you can even get as granular as a model of a car, so you could really get very personalized experiences for your customers, if you really personal those things out, so I just wanted to call that out.
Frances: That is fabulous.

Purna: Those points are so spot-on. You understand your audience, and then you can use your ad and side links to shorten the conversion cycle for them. So it’s like, “Great. Just come here and estimate your payment,” or “Come here and schedule a test drive,” so help it, make it easy to take the action that you want. So another use that I really love for remarketing, and I’ve used this in the past, is up-selling to recent converter.

The big mistake several of us make is we’ll take our people who’ve already converted, and we’ll just bid mod. them (-90%), so then we just exclude them. The lightest ones in us to buy from us, they’ll probably buy from us again. And that’s what we want to do with that. So I guess the good old MacDonald’s Events Prize was that. So they came, they bought the shoes, may be they want the sock. They came, they bought the cars, may be they want to come back and get some new wheels or like a different service plans from you.

There’s always different things that we can present to them and try to improve customer lifetime value and grow it that way. Like you want to build the bonds with the customer for as long as you can, so you keep them with you and not to your competitor. So how do I do that? I want to try to offer very relevant cross sells and try to be really helpful. So some exclusive discounts, just like, “Hey, I noticed that you have bought this. Do you know that if you bought this red lipstick, do you know that there’s other red lipsticks on sale for like 20% off?” Or, “Hey, you’ve come here for an oil change. Do you know if you buy another two oil changes now, we’ll give you 50% off?”

The different purchase incentives or gifts with purchase; it’s really a good way to start thinking about it. Starts low-hanging fruit. They bought from you once; we can get them to buy again. All right. And then my final list of five is, I call it “To Infinity & Beyond” – this is the rubble list, right? Why should we let the constraints of our own website define us? What we really want is to think beyond our own site and get onto OPAs which, pretty much the same Greek sound of happiness, we want to grow with ‘Other Peoples’ Audiences’, so that we can grow our remarketing pool.

Remember we need at least a thousand cookies in a list before a list is active. So let’s see how can we grow our remarketing list and pool. How do you do this? Try to find mutually beneficial relationships.

So if I can talk to a person who is targeting my same audience but is not a competitor; for example, if I’m selling ‘Petting Chimes’ and then I have another person who is selling, like, ‘Pet Clothing’, we’re not competing with each other, but we are selling to the same audience which is “pet owners.”

So that’s something we can work at that, “Hey, I’ll put your UET tag on my sites, and I’ll give you my code on your sites, and then let’s see if we can grow our list that way.” It’s a very, very interesting way to make remarketing even more efficient. How do you do that? So you want to tag your related complementary websites – you’re going to work that out yourself, that’s not something we can help you with. But we can certainly guide you on how you tag the website.

Then you want to bid on broader keywords; remember they may not have been to your website, may not be as familiar with you, but they could appreciate your product. The “Bid on Product Keywords” here, and go really generous with the incentives and the special offers and try upping the bids, may be round about 50 to 150% mark. Pretty cool, right? And, hey Frances, what do you think? Think we can earn our bonus or that promotion that we wanted?

Frances: Absolutely. We give a very ingenious way of leveraging customers that, you know, have that synergistic interest. Yeah, and I think we’ll actually talk a little bit more about that and how the Microsoft Store used strategies like this in order to really achieve their aggressive goals. So just to recap that list that Purna went over, we talked about window shoppers, we talked about attracting the abandoners – we know that 70% of your shoppers are going to abandon that cart, doesn’t mean that they’re not still interested, doesn’t mean we shouldn’t still go after them. We talked about reengaging those people who are already engaged; don’t just ignore them because they’ve done something in the past.

And in that same vein, you’ve got to up-sell to those converters. If I bought… someone mentioned high ticket items… if I bought one high ticket item previously, I’m probably more likely to buy in that same luxury goods category again. Or, like Purna mentioned, if I bought one item as part of a group of items, like I bought running shoes, I probably want to buy the latest running sock or the latest running top.

And then the OPA, the ‘Other Peoples’ Audiences’ – so this is where, again, you get pretty strategic and digging into that data, mining the information of the people who are coming to your site. But with all that being said, we do know that at times remarketing can come across a little creepy.

I think probably when it first started, all of us had this story; all of us had a friend or a family member who has asked us, “What is it about these ads that follow me all around the internet? So weird. How does Facebook know that I was looking at a pair of shoes on Merrell’s store? And how does Merrell’s store know that I’m now on Facebook, and showing me that ad?”

And I think that that worry has decreased a little bit. I think that people are much more used to seeing it. But there are some ways that you can make it a little bit more comfortable for your shoppers and for people hanging out and searching on the internet. So, avoid being too specific in you ad copy. If you’re going to customize ad copy to a certain list, you know, be subtle, be helpful.

Again, going back to that notion we talked about in the very beginning, you’re having a one-on-one conversation with this customer, so be as helpful as possible. Use product category so, I, as the shopper or the searcher, can relate it back to you. “Oh, well, that makes sense because I was looking at those sandals last week. I just hadn’t made my mind up about them yet.”

And then, testing varying cookie windows, so you might want to retarget me; if I looked at a pair of sandals on Nordstrom, they might want to retarget me for may be the next five to seven days, but then it really might not make sense for them to keep sending me those images or those advertisements about those sandals a hundred days later. That’s probably when I start to think like, “Well, obviously, I didn’t buy them,” or, “I already had bought them when you started coming after coming after me.”

So, look at how long you use those cookie windows for.

And we’d just like to cover quickly some emerging research. So before we go into what the Bing Ads Product Team has done, some internal research on, we also have some other research that we want to highlight for you. So here are some guidelines to help you. The best way to think about this is, MJ already mentioned this, but you know you have a conversion funnel or a conversion path with your customers; they’re going to come to your Home Page, then they’re going to end up on your Category page, into your Shopping Cart, and then the Confirmation page. You know that that’s the road you want them to travel when they’re on your site. These are some guidelines, and sort of, best case practices to try with your bid modifiers.

So, if you have a remarketing list of all of those people who have come to your Home Page but not looked any further, and you want to target those people again, up your bid modifying by 50% and keeping that same keyword, ‘Running Shoes’, right? You might be be Brooks.com, you might be Nike, or you might be a local sports store. These are guidelines on the best way to increase and manage your bid modifier for that customer funnel. And some other looks into when to increase your bids.

So, obviously you want to target the most valuable prospects who had shown interest, so they come to your website, they… Some of the things you might want to take into consideration at this point when you’re retargeting is broadening your keywords. So what are other keywords that they’re probably searching in that category where you’re really going to want to recapture their attention and look at seasonality? So, a really good example of this is golf balls.

So, more golf balls are sold leading up to Christmas and leading up to Father’s Day, because there is an easy present, for your dad or for your uncle or for your grandfather or your grandmother if, you know, if they’re all living down in Flordia, they are probably playing golf all day long or at least I hope that that’s what they get to spend their time doing. And so, you know, you might not want to retarget that person who has bought golf gifts from you, leading up to Father’s Day or Mother’s Day, or leading up to Christmas and the holidays in August or in February. So take that into consideration when you’re looking to increase bids.

Same, when you’re looking to decrease bids, these tend to be the users who don’t deliver value or who are not likely to convert again. They just want to take a look at your website. Something to always keep in mind, I love this example of career-page visitors. So, at Microsoft, we get a ton of people who come to our career page that probably look like good visitors – they come to the page, they spend a long time on our site [chuckles], they look at multiple pages – but if we actually dig down into our analytics, we will see that they’re spending most of their time on our company culture page where the career page or looking at where we have offices, those are people that we need to be showing ads to, to try and find something on the Microsoft Store.

These are people who clearly want to come and work for Microsoft, and they’re a different type of user. So you’re just keeping that in mind when you’re mining, really get a good insight into who’s coming to your site and why they’re coming there. So here’s some best case practices on if you’re customizing your ad, so this is kind of, say, you’re not doing bid modifiers, or you wanted to bid on [inaudible 0:34:44.2], first customization, these are some terms and action items that are really spectacular to different stages of your customer funnel, so something to really think about.

In traditional campaigns you might just have your keyword ‘Sports Apparel’ for your remarketing campaigns; you’re probably going to want to broaden that, and you know, if you’re trying to then keep up to a product page, you’re probably very specific in a traditional ads. We don’t need to be that specific in your remarketing campaign, because they’ve already been there, they already know what you’re selling, and you can still keep your keywords a little bit broader and therefore capture more people’s interest.

So, using broad match, looking at different sets of keywords, don’t be constrained in what you might have to, you know, gone very specific or very narrow on a traditional campaign, you don’t need to do that with your remarketing campaign. Purna, did you want to add anything here? Nope? Okay, perfect. So, just to go over your checklist, so, everything that we talked about, that Purna and I talked about, these are just the four things you need to keep in mind.

And you can go back to these slides, you can download this deck and you can go through it in more detail, but make sure that your UET tag is placed everywhere across your site, make sure that that code has been copied correctly, so you don’t want to think that it’s there and you’re collecting people for your remarketing list and then all of a sudden, a month into it you realize the code is [inaudible 0:36:27.0], so double check that.

Double check your audience list for rules and errors; so this is just making sure that you click the right category with the right rules in place, with the type of audience you want to connect. And then, if you are using a tag manger, the Bing Ads UETs things that all of the typical tag managers that are out there, make sure that that tag is set to fire correctly within that tool that you’re using so that you know you’re collecting the right information. So I’m going to kind of now… So these are kind of the remarketing strategy and set up.

So, once you’ve got everything in place with your tagging and the technical side of that, decide on how you want to mine your audience, and start building those lists. So again the best place to start is your conversion path funnels, start building those lists of them, like the list that Purna talked about, you can get really creative in how you want to build other lists as well. And this is when you’re really going to start seeing… So you need a thousand people in your list to really start remarketing to them, so know that it might take a month or two for you to build that up, which is why we suggest you start doing it right now, because you’re going to want those remarketing lists in place when we get into busy shopping seasons like back-at-school and the holidays which aren’t really that far away in PPC language [chuckles] even though they’re still far away in regular human time; PPC time is faster.

And so, to really talk about how you can test your remarketing performance, I want to introduce Christopher. Christopher’s on our product team here at Bing Ads. They’ve run numerous studies, experiments and tests, really sophisticated modeling, and what we’re here to talk today is how you can take some of that and do it at home, and really get it into sophisticated remarketing performance. So Christopher, I’m going to hand it over to you.

Christopher Walleck, Program Manager, Bing Ads, Microsoft:

Christopher: Thank you, Frances. My name is Christopher Walleck and I’m a Program manager on the Bing Ads Network Management Team. You’ve heard a great deal today about how remarketing works, how to implement it and today we’re able to share with you the results that we’ve seen from our remarketing experiment. So first I’m going to tell you a little bit about our team, the Bing Ads Network Management Team, then the experiments and then talk about how you can perform your own experiments. Very briefly, our team, the Network Management Team has a three-fold charter with one [overall 0:39:02.6] mission. Optimizing advertiser performance, and growing our business.

What this means is that on a day-to-day, we look at how the different products and features we offer affect the auction market place. We see, for number two, ‘business planning & forecasting’, how we expect these products to affect our trajectory, going forward and for number three, we are an extremely quite a heady team. We share engineering data with other teams and we use this data to look at how our …performance is changing. So let me tell you a little bit about our remarketing experiment.

Now, when you’re implementing remarketing, you may ask the question: “Okay, what kind of difference will this make ‘pre’ and ‘post’? What if I just run the experiments? What if I set up remarketing for a week and then tests up to week two and see what the difference is?” But the critical issue here is you want to minimize the potential differences between your two samples and let me tell you how we minimize that difference.

Within our experiment, we have a ‘treatment’ and a ‘control’. If you have ever done any kind of A/B testing, you can think of these as your ‘A’ and your ‘B’. The difference between our treatment and our control with – one, have ‘Remarketing bid boosts’ enabled for treatment, one, have ‘Remarketing bid boosts’ disabled – the control. This is the only difference between the two samples and one thing I want to call out here is that we did this at the ad group query level, that is one ad group matched to one query, so we got to the most granular level possible in order to see how these results will affect you. And the results are exciting.

When we compared 2.3 million remarketing impressions from 50,000 different DSQs, distinct search queries across 470 advertisers, small, medium, extra-large, across 18 different verticals, from auto to travel, what we saw is that remarketing positive bid boosts, positive bids drove more user engagement, drove more clicks, and drove more conversions. Let me talk this through for a second. You can think of these numbers as the remarketing three, four, five. In aggregate, remarketing led to 3% increase in average dwell time, that is how long do users spend on your page, after clicking on your ad?

In terms of overall clicks, remarketing also led to 4% in aggregate more clicks, from users that are high value. They have been to your site; they have expressed interest in what you are offering, and what’s most important for you, your CVR, your remarketing conversion rate. Remarketing users are more likely to convert and overall, the remarketing rate for advertisers that implemented this at the ad group level was 5% and this is just the beginning. Let’s tell you a little bit about the advertisers that got some extra special performance from remarketing.

Frances: So this is Frances again, and I just wanted to kind of recap what you just said. So when you looked at a huge volume of advertisers and searchers, those advertisers that had turned on, not just remarketing, but also bid boost, so they had increased the bid that they were…or the amount they were willing to pay for the ad to be shown to a person a second time in that remarketing list, you saw more user engagement, more clicks and more conversions.

Christopher: That’s absolutely right. This is across 2.4 million ad impressions, and when we saw positive remarketing bid boost, they just drove, as you said, better engagement, better clicks, better conversions. Some advertisers saw up to 30% more clicks, a 20% lower click back rate, the click back rate being the proportion of users that navigate immediately away from the page – that means that you get the users you want to stay in your page – and up to a 15% higher conversion rate. We saw this in Travel, Sporting Goods, Home & Garden, across different verticals in the experiment.

Now, you may say, “This is all fine and dandy, but how do we run our own test? How do we…? You know, we don’t have this experimentation technology but we want to see it, if remarketing works? We want to get it to work for us.” So, now I’ll tell you a little bit about how to test it yourself. So the first step is, if you haven’t yet, you want to configure a new remarketing list. We’ve talked about how to set it up; we’ve talked about the different criteria to keep in mind. So, once you set it up, make sure it hits the user minimum we have first; this is in place such that you’re able to preserve the user anonymity. If your remarketing list only had one user in it, then you’ll be able to target that person specifically, so wait for that to happen.

Then you want to set up two different test ad groups that has the same keywords, but at different bid boosts. And these will be your ‘A and your ‘B’, your treatment and your control. So try for instance, 0% and 50% for your first go. Then, this is the cool part, and we’ll show this to you later, what you want to do is use ‘Day of the Week’ targeting in Bing Ads so that you can disable and enable your ad groups every other day. So it’ll go Monday, Wednesday, Friday for one, and Tuesday, Thursday, Saturday for the other. This will allow you to control for the seasonality that comes from day-of-week traffic. What if Monday traffic is substantially different from Tuesday traffic? And as you run this experiment, you will be able to minimize that effect.

And so finally, you want to run this experiment for four plus weeks, as long as you can. And then you’re going to use a t-test to evaluate the statistical significance. And if you remember, if you’ve ever taken a statistics class in high school and college, what a T-test does is basically determine if two sample data sets are substantially different from one another. You may remember a Z-test; basically the same thing, but t-test is used for a smaller sample size. And if you want to learn more, we have all kinds of information on our page that you can look at.

Now, what you’ll have in your template is the clicks for one remarketing bid boost and the clicks for the other remarketing bid boost, and using this test, you’re going to be able to validate the statistical significance of these results. Often we may look and say, “Oh, look, it’s a 10% difference between these two!” But is it statistically significant? And you’ll see in the template that we’ve provided that you’ll be able to input these numbers and it’ll directly tell you if the results are statistically significant. The value next to the t-test on there is the P-value.

And you’ve probably heard about P-values before. If you haven’t, the simplest way to explain it is that this number, on a scale of 0 to 1, is what is the likelihood that these results came from chance? What is the likelihood that they came just from chance alone, and not from being different? So the smaller, the better. And for this we have it directly configured, so if the P-value is less than 0.1, then it’ll show as statistically significant. Don’t worry about the numbers, just worry about putting in the different clicks from your performance, and it’ll tell you if what you’re seeing is special.

Frances: This is Frances again, and essentially if you’ve run this test for, say, a month or two months, you drop your numbers into the template, it tells you this is a statistically relevant result, what it’s telling you is you should really be implementing a bid boost strategy because the results that you saw were favorable and also are likely to be favorable across multiple, say, ad groups, multiple campaigns, not just within that one test that you did.

Christopher: Absolutely. And then, the next step. Once you’ve done the first test, with say 0 and 50%, what you want to do is repeat the test with different bid boosts. You want to find the best bid boost for your campaign that optimizes your performance. So once you’ve tried 0 and 50, try 25 and a 100, try a 100 and 200, you want to find the best bid boost for your ad groups, and then, as we’ve talked about before, you want to test different membership durations – that is, how long are these users going to stay in the remarketing list? Try one week versus 30 days. And you can use the same method we used before, and in this case, you have two ad groups and the same bid boost.

Because you may be able to see after 30 days, “Wow, this group of users that were in the 30-day remarketing list, actually, the conversion rate overall is higher,” and you’ll want to know what combination is best for you. So, test different bid boosts, test different membership durations, and start today. One last note: Make sure when you’re doing the day-of-week targeting, that the end of the week, you switch it. That way it’ll go Monday, Wednesday, Friday, Sunday, and then the next week, it will switch. Otherwise you’ll have two Mondays in a row. So, when you’re running your experiments, make sure that you can figure it so that you can get the most variety possible, and then you can see how to get the best remarketing performance you can.

Frances: Great, that’s amazing. Thank you so much for sharing, you know what the product team found and the results that you saw. For some people on the webcast, this might seem too sophisticated for your business; for some people this might be exactly what you’re looking for, to really find out, like Chris said, what is the best combination between bid boost and membership duration. And I think what it really goes to show is there are a myriad of possibilities out there that are going to help you drive revenue and attack really strong or ambitious goals that your business or your company have.

And to talk about that, I want to introduce Ben and Natalie to talk about how Microsoft Store, Microsoft.com and their store – so this is where we sell everything that Microsoft makes, both hardware as well as Office 365, other software; I’m talking about how they use remarketing in their business.

Ben Carson, Global SEM Lead, Microsoft Store, Microsoft:

Ben: Thanks Frances. This is Ben Carson, and sorry to disappoint a few of you guys that have already written in here, I’m not the Presidential candidate although I’m not ruling that out for, you know, the future. I am the Global SEM Manager for Microsoftstore.com. So as Frances mentioned, everything that Microsoft makes in terms of Xbox, Surface, Office, we’re driving everything to microsoftstore.com. So the question that we brought to Natalie and our agency Point It is, you know, where are the pockets of untapped opportunities for Microsoft Store?

It’s one thing to drive revenue, which is something we’re all interested in, but Microsoft Store specifically optimizes to a ROAS, which is Return On Ad Spend and efficiency target. So, not only where the untapped revenue opportunities, but where are those pockets of high efficient spend that will still drive revenue for us? And so that was the question that we came to Point It, our agency, to help us answer. And so the solution we came to was the Bing UET implementation. And I’m going to let Natalie talk a little bit more on how we leveraged that.

Natalie Barreda, Senior Client Manager, Point It Agency:

Natalie: Thanks Ben. So, just to kind of give a little bit of background. Point It, the agency that I work with, and Microsoft Store have been working together for many years. So it’s been a long-standing relationship and at this point, we’ve felt how, you know, we’ve campaigns fully built, how we’ve really kind of saturated you know, kind of, the lower part of the funnel which, you know, kind of what we’re looking for. So you know as Ben said, we’re held at a fairly aggressive efficiency target, so we were, you know, as he said, we were tasked with trying to find where can we find additional revenue but still maintain that efficiency.

You know, as I’m sure, a lot of you know, you know, Google has been using their RLSA for some time now and you know, typically you see really good results with that. So as soon as Bing came out with the Universal Event Tracking, it really gave us an opportunity to try to capture any additional revenue that we were able to get, that was may be being left on the table a bit. So one of the things, you know, as I mentioned before, Google has the RLSA capabilities for some time, but one thing that really makes UET special and kind of stand out from Google which has really been a big part of why we’ve kind of put all of our eggs in this basket is with UET, and we’ve mentioned this before, we’re able to use those OPA audiences.

So those Other People Audiences, which I know may seem a little….you know, there’s some kind of questions and stuff like that around sharing that kind of data and stuff like that, but really what you’re doing is you’re finding opportunities for you to provide a more personalized experience for folks that are going to be relevant to your site. So one thing that, you know, in our experience at Point It, we tend to and understand at Point It, we tend to work with enterprise-level client, so, you know, so someone such as Microsoft has a ton of different properties and stuff like that that allows us to target our ads to people who are….target and message to people who are showing interest in things such as Office, Surface, things of that sort.

So you are….You know, it’s folks who’re visiting those Office sites, you know, doing some research on some of the Office products, may be Power Point and things like that, so they’ve already expressed interest in this, so all we’re doing is that we’re showing them ads that are going to be more relevant to them. So, we’re going in and calling out… obviously they’re interested in, you know, the capabilities that Power Point has, so you know, what we’re doing is, we’re going in and showing them ads that really speak to those capabilities and kind of, you know, trying to give them the best experience that they can get.

Ben: And I just want to chime in quickly here, because we did see a question come in about, “Doesn’t this violate privacy or what are the concerns there?” I just wanted to reiterate that there is no personally identifiable information in these remarketing lists. So as you are partnering with other companies, or within your vertical, and sharing, you know, the O-P-A, or is it the…?

Natalie: OPA [chuckles].

Ben: OPA? Yeah?

Natalie: Yeah.

Ben: Is that an acronym we’re trying to start ____?

Natalie: Yeah.

Ben: Yeah? I love it, your OPAs. Just know that there’s no sensitive PII information that’s shared there, so I just wanted to call that out specifically because we did see some questions come through on that.

Frances: So I think there are a couple of ways that people can really take advantage of that OPA list. This is Frances again. So one example we saw with small medium businesses that we work with are, you know, a local store that might sell pet food and pet supplies – like a Mud Bay here in Seattle is a store that does that. They might want to partner with another local store that does doggie daycare or doggie grooming or sell doggie outfits. There’s another company here in Seattle that sells beer and pretzels for dogs.

So, you might want to partner, and if you’re a local business, or you’re in the same chamber, you might want to partner and share some audience remarketing lists just in the way in a big enterprise corporation, like Natalie was talking about at Microsoft, we might want to share people who’re buying X-box games might really also be interested in buying PC games.

Now we’ll probably target them differently, they sit within different groups at Microsoft, but that doesn’t mean they’re not interested in… If they’re interested in gaming, they’re probably interested in similar type of products. So Natalie, I’m going to hand it back to you to talk through the results that you saw specifically with Microsoft Store with your remarketing campaign.

Natalie: Awesome. Thank you, Frances. So you know, as we’ve mentioned before, we’re really just trying to drive a personalized experience for searchers. So, all in all, and when you’re comparing our Bing Ads remarketing to our …efforts that we’re doing in Google, we’re seeing a higher order value, so people who are converting are purchasing things that are a little bit higher ticket, so great things to see there.

Additionally, we’re seeing higher ROAS compared to adwords,

so as Ben said before, we are really focused on efficiency, so not only are we bringing in additional revenue but we’re bringing in additional revenue if it gives some more efficient….

And then, also we’re seeing lowered CPCs compared to what we see with ad words; you know, these are things that really kind of contribute to that ultimate bottom line of trying to drive a more efficient campaign, so, you know, being a little bit smarter with how we’re bidding and being able to drive those lower CPCs, you know, ultimately helps towards our bottom line. And overall we have account-wide, if you’re looking at the two engines, we’re seeing a 20% higher average ROAS.

Frances: That’s amazing. So Ben, you’re happy with these results [chuckles]?

Ben: Yeah, we’re extremely happy, and we’re rolling it out globally as well as we’re seeing results, continue to demonstrate this high level of efficiency within US.

Natalie: Yeah, yeah, I know, absolutely. And it’s not kind of something where we just set it and we’re just going to kind of let it go. There’s also opportunity to, like, expand out how long people are on this audience list; you know, testing out different audiences, compared to the audiences that are running now. So, it’s definitely a continuous learning process, but a very efficient learning process, which is good.

Frances: Yeah, I think all of us who have worked in PPC or SEM for a long time know that it’s never set and walk away as constantly….really, constantly optimizing, constantly tinkering, and constantly running experiments like what Christopher and his team do, all of which is trying to find that next level of efficiency. So I want to thank Natalie and Ben for coming and sharing your experience with us. I really want to thank Christopher for talking about how the Products Team is looking at experiments that you can take and do at home. And something to leave everyone with, I really encourage everyone to get set up.

If you have questions, you can go to BingAds.com/UET – there’s videos, tutorials there – and get set up, so that you can start creating those, your lists, testing your bid modifiers, and testing the length of your cookies, and asking us your question. And here are some of the best ways to ask us your questions and stay in touch. So, there’s a Bing Ads Blog, like MJ mentioned, we’ve got through a lot of people’s question built in the chat window and talking, but I’ll be writing a blog post; it will go live probably tomorrow or the next day answering everyone’s question that we didn’t get to, so check that at BingAds.com/blog.

I already mentioned the UET website. And then, if you’re online, either Facebook, Twitter, Instagram use the #AskBingAds with any questions that you have about remarketing, UET or anything to do with page search, and we will answer you there. And we also do monthly video blogs answering questions. And I’m going to hand it back to MJ to close this out.

MJ: Yeah, thank you so much everyone. This was really, really engaging, and important information. There was one question that I did want Natalie to answer from her perspective. And that was, is the only difference between remarketing on Ad Words and Bing Ads OPA? Is that the only difference or the only advantage of running on Bing Ads remarketing is OPA?

Natalie: That is one of the biggest differences that we take advantage of for, but I think towards to your bottom line is you see super, you get to see super CPCs, so ultimately you know, as they said before, like that’s going to add feedback into your ROAS, Return On Ad Spend, whatever, especially if you’re doing… or even CPA type of thing. You know, it’s going to go back to your, like, key metrics. So like cheaper CPCs, the OPA audiences…

MJ: And do you find that the audiences seem to respond in a more, you know, a higher quality of [close rate], if you will….?

Natalie: We see fairly comparable, like, click-through rates, fairly comparable conversion rates, so… But this is already saying, that like, RLSA tends to have higher click-through rates, and tends to have higher conversion rates. So, like for me the biggest reason that we wanted to use it is not because it’s incredibly different from Google; it’s because there’s money on the table, and we don’t want to leave it there.

MJ: Awesome. So, take that away, don’t leave money on the table, and definitely take advantage of remarketing in the audience that you’re not reaching on Google, on Bing. And please remember, join us May 31st, we have another awesome webcast coming up about ‘The Science of Data Feed Optimization for your Shopping Campaigns.’ It’s definitely going to get into the important aspects of managing your data feeds for shopping campaigns and really optimize your efforts in that area. So thank you everyone for joining us and until next time, I hope you have some really fantastic and well-performing campaigns. Talk to you soon.

Natalie: Bye everyone, thank you.

Related Articles:

• [Transcript] Bing Ads Advertiser Science Series – The Kevin Bacon Approach to Keyword Attribution
• [Transcript] Bing Ads Webinar – Advertise Globally with Bing Ads
• Bing Ads Advertiser Science Series: The Science of Brand Bidding Webcast