James Marchant, Author at Conversion.com

Migrating to Optimizely X Experiments from Optimizely Classic

With the release of Optimizely X and the new Optimizely X Experiments, many Optimizely users are wondering how they can migrate to the new platform and what it means for their tests in the old platform (Optimizely Classic). As an Optimizely three-star partner, we get access to Optimizely’s newest products before release, including Optimizely X Experiments. We can prepare ourselves and our clients for big product updates like this. Armed with this knowledge, I’d like to guide you through the process of migration and answer a few common questions.

What’s changed and why should I migrate?

The new platform delivers some great improvements as well as some brand new features which will hopefully enhance your testing. Here are some key features and changes that Optimizely are bringing to the table in Optimizely X Experiments:

  • A new visual editor that loads faster and provides the ability to view websites in different responsive states;
  • An updated code editor (gone is the old code engine) that has a separate section for variation CSS, new utility functions and the ability to control when variation code is loaded onto a page;
  • Two new features: Pages (a templating system that combines URL targeting and Activation modes) and Events (an updated version of Goals);
  • A new Results page with an updated stats engine (each Metric now shows “confidence intervals” rather than “difference intervals” and the ordering of the Metrics affect how quickly they will reach significance);

A burning question for some is: why should I migrate over to Optimizely X Experiments? While there isn’t a golden reason why you should, there are a few reasons you might want to consider:

  • Code editor improvements. With visual editor changes no longer generating code and the task of deciding when code should execute is not down to Optimizely but the developer, code can be cleaner, easier to read, and potentially easier to implement within the new platform.

How do I migrate to Optimizely X Experiments?

Before you start the migration, you first need to evaluate the status of your tests within Optimizely Classic:

  • Do you have tests that are still running and haven’t finished?
  • Do you have any winners that are running which you haven’t implemented fully onto your site yet?

The answers to these questions are crucial, as there are a few features from Optimizely Classic that cannot be carried over automatically to Optimizely X Experiments:

  • Experiment code
  • Experiment results
  • Goals

Thankfully, Optimizely allows us the option to run both Optimizely Classic and Optimizely X at the same time. This means that if you answered yes to one of the above questions, you can combine them in one snippet (a soft transition) without worrying about experiments being paused/disabled.

Optimizely Snippet
This screenshot from the settings panel in Optimizely X shows Optimizely X being enabled, as well as both Optimizely X and Optimizely Classic set to load in the same snippet.

Enabling both at the same time, however, does come at a cost: an additional 50KB is added to your snippet when using this option. This could affect the speed at which Optimizely loads your tests – and impact performance. You will therefore need to weigh up the advantages of this soft transition and the slight disadvantage that comes with the additional snippet size.

If you don’t have any active experiments or winners, then you can enable Optimizely X and simply select the “Use only Optimizely X” option in the Snippet Configuration (a hard transition). This will mean that all of your Optimizely Classic experiments will be disabled (but still accessible).

What if I want to move my existing Optimizely Classic experiments to Optimizely X Experiments?

Unfortunately, there is no automatic way of moving your experiments from the old platform to the new one so you will need to manually do this. There are a few things first, however, that you will need to make a note of for each of your Optimizely Classic experiments in order to move them over to Optimizely X Experiments:

  1. Experiment Name (if you want to keep the same name)
  2. URL Targeting
  3. Activation Mode
  4. Audiences used
  5. Goals

When creating a new experiment in the new platform, Optimizely requires the following information:

  1. Experiment Name
  2. Page(s)
  3. Audience(s)
  4. Metric(s)
  5. Variation names and distributions (optional)

The Experiment Name will be the same as your old experiment name that you had before.

Pages is one of the new features in Optimizely X Experiments (as stated above) which combines URL targeting and the Activation mode type. Both URL targeting and the Activation mode function in the same way as in Optimizely Classic, so it should be pretty simple for you to carry them over.

Page creation window
The Page creation window

Audiences are automatically available between Optimizely Classic and Optimizely X Experiments, so all you need to do is add them into the experiment.

Old Audiences
Old Audiences
New Audiences
New Audiences

Metrics are the events you want to use to measure your experiment. Events are an updated version of goals and creating them is similar to before. You may wish to read Optimizely’s Knowledge Base article on them to familiarise yourself on how they work.

Finally, Variation names and distributions can be added/changed depending on what your old test had.

If your test is a multi-page test, then you will need to create separate Pages for each of the Sections in the experiment and apply them to your new experiment. Within the visual editor, you will then have the ability to switch between the Pages that you’ve applied to the experiment and then add the appropriate changes/code in the editor.

Switch Pages in the Editor

As of writing this, there is no support for multivariate tests nor dimensions (so you will not be able to apply advanced segmentation to your tests). We expect this to be added relatively soon.

Finally, with the new platform comes a complete rethink of how you have to code your experiments which, of course, presents a challenge in porting over experiments. (If you are not a technical user, and don’t have knowledge of Javascript and jQuery, then you will need to utilise your nearest friendly Front End Developer to help you!).

Optimizely Classic required you to write your code in a specific format in order to use Optimizely’s algorithm so that it could execute as fast as possible as well as utilising force parameters and custom functions. In Optimizely X Experiments, you no longer have to write your code in this way but you do have to manage the timing of your own code as all variation code is executed immediately before the page has loaded. Don’t be alarmed though – Optimizely have provided a few useful utility functions to help with executing code at the right time. Converting your code from the old form should be fairly simple. As an example, take this code from an experiment in Optimizely Classic:

// Update the CTA text

$(‘.cta’).html(‘Click Me!’);

And now we can see how that same functionality is done within Optimizely X Experiments:

// Import the utils library
var utils = window.optimizely.get('utils');


// Wait for the element
utils.waitForElement('.cta').then(function(element){
  // Update the CTA text
  element.innerHTML = 'Click Me!';
});

As you can see, I am using the waitForElement utility function in place of the jQuery selector that I had before. This function will wait for the element to appear within the DOM and then execute the promise function. You can repeat this process for each line of code that is outside of the force parameters.

If you want to access the jQuery that is bundled in your snippet, then you can import it in a similar way to the utility functions:

var $ = window.optimizely.get('jquery');

Lastly, if you had any CSS that was injected via Javascript/jQuery within your experiment, you can now separate this into the new Variation CSS which will make it much easier to create and edit your styling.

Variation CSS in Editor

And that’s it! You should now be ready to publish your experiment to the world.

If you are using our free Optimizely Chrome Extension, then you will be happy to hear that we have updated it so that it is fully compatible with Optimizely X Experiments.

Do you have any suggestions or tips/tricks for migrating form Optimizely Classic to Optimizely X Experiments? Please share by leaving a comment below.

6 Essential tips for any developer using Optimizely

Developing within Optimizely is a unique undertaking that has few parallels with conventional front end development. In this post I will outline six gems of knowledge that I have gained while building tests in Optimizely for a wide range of Conversion.com clients. Please note: this post assumes that you have knowledge of writing code within Optimizely.

1. The Optimizely log is your best friend

My first essential tip is the use of the invaluable Optimizely log. The log contains all the information on bucketing, segmentation, audiences and code execution on your site on page load while also displaying execution times of each part in the process (for further documentation on the log, see this Optimizely knowledge base article).

To access the log, you simply type the following into the console of your page:

window.optimizely.push(‘log’);

this will then return something like so:

optimizely-log

This is extremely useful to locate any code that is preventing the rest of your test(s) from running that in-turn will increase flicker for the user. As all code must be written in the Identifier/Action format (see this useful knowledge base article from Optimizely on how code is executed within tests), it can be easy to forget to add non-optimised code in your experiment. Here is an example of code being delayed because it doesn’t follow the correct format:

optimizely-log-code

The code highlighted is a simple variable declaration which unfortunately does not follow the correct format. You can also see, on the lines preceding, that Optimizely is continually waiting for the document to be ready in order to execute the code (our example shows it took 617ms from encountering the code to the document becoming ready, however on slower sites this can take much longer).

You will inadvertently run into this when writing tests, and it is useful to use this tool to check that all of your code is compliant in order to reduce any possible flicker for the user.

2. Use custom jQuery functions in order to add non-compliant code

There will be times when you need to add code that won’t be in the right format and there’s no way you can transform it. When this issue arises, you can simply create your very own custom jQuery functions in order to bypass it. To do so, you will need to first define your custom function within the Optimizely force parameters (see this knowledge base article on force parameters) and then reference the defined function in your code, e.g.:

/* _optimizely_evaluate=force */
$.fn.customFunction = function() {
    var number = 10;
    for (var i = 0; i < 10; i++) {
        number++;
    }
    $(this).append(number);
};
/* _optimizely_evaluate=safe */
$('.main').customFunction();

In this example, we have defined a function originally named ‘customFunction’ which increments a number ten times before appending it to the subject when called. The ‘customFunction’ is then called on the element(s) with the class ‘main’.

The potential applications of this method are large, with other examples ranging from timeouts to loading external scripts. Another benefit for this is that the function needs to be called using a selector that is run within the Optimizely code engine, this can be then utilised in order to check for when that specific element is available to modify/change.

3. Use ‘onmousedown’ events for AJAX loaded buttons & outbound links to track goals inside Optimizely

Most Optimizely tracking goals can be added via the “Create Goal” window within the editor, however there may be times when you need to use custom events in order to record clicks on buttons & links.

When a click goal is added within the editor, Optimizely adds an ‘onmousedown’ event (see Optimizely’s knowledge base article on click goals) to the specified element and this is attached after all variation code has been run. The reason for this is so that it can more accurately track elements that may send the user away from the page.

In cases where you want to track an element that may be pulled in via AJAX or you want to manually track outbound links, then you can use the same event listener on those elements. For example, if you wanted to track a link that goes to an external page, you could do something like this:

$('.link').mousedown(function(event) {
    window.optimizely.push(["trackEvent", "eventName"]);
});

If the element you want to track is loaded via AJAX (or it loads later after page load), then this method poses a problem as Optimizely will continually poll the page until document ready, at which point it will execute the code regardless of whether the element is there or not. To get around this, you can use the .live() jQuery function (this is currently deprecated in all new versions of jQuery and replaced with ‘on’, however Optimizely uses the older 1.6.4 version that has this available) within the Optimizely force parameters like so:

/* _optimizely_evaluate=force */
$('.link').live('mousedown', function(event) {
    window.optimizely.push(["trackEvent", "eventName"]);
});
/* _optimizely_evaluate=safe */

This code will then execute on all elements with class ‘link’, regardless of when they get loaded onto the page.

4. Use the force variation parameters to bypass audience and targeting settings

When developing a test, you will want to check your code in as close to the environment that a user might see as possible. While the visual editor and the preview modes are convenient to use, they do directly modify the page in a way that could potentially affect the test and you are of course not seeing it in the same environment that the user will see it in. Fortunately, Optimizely allows you to force tests to appear regardless of their audience and targeting settings.

Firstly, you will need to make sure your Optimizely project allows you to use the force variation parameters. To check this, simply go to Settings->Privacy within the dashboard and make sure “Disable the force variation parameter” is unchecked. If it is checked, just uncheck it and hit “Save”.

force-settings

Once that setting has been updated, you can then go to your site and add the following URL parameter:

?optimizely_x[EXPERIMENT ID]=[VARIATION INDEX]

Just replace “EXPERIMENT ID” with the ID of your experiment (you can find this in the editor URL for your experiment under the parameter “experiment_id”) and “VARIATION INDEX” (this is a zero based index where 0=Control, 1=Variation #1, etc.) so that it looks something like this:

http://domain.com/?optimizely_x2850651259=1

If your test is a multivariate test however, then you will need to use a slightly different syntax:

?optimizely_x[EXPERIMENT ID]=[SECTION 1 VARIATION INDEX]_[SECTION 2 VARIATION INDEX]

As you can see, you separate each section variation with an underscore, where

http://domain.com/?optimizely_x2850651259=1_0_1

will show variation 1 of section 1, the control of section 2 and variation 1 of section 3 of your test.

Always remember that this method doesn’t check for whether your test will run under certain conditions, and should only be used to check if your code works. For more information on force variation parameters, check out Optimizely’s knowledge base article on them.

5. Check the revision of the Optimizely snippet to ensure you are seeing your latest code

To ensure speed and reliability, the Optimizely snippet is loaded using Akamai that, like all other CDNs, will evaluate the best server on their distributed network to serve the code with (generally this is the closest geographically). This however comes at a price of increased save times due to the nature of invalidating files on CDNs. I’m sure many of you have had to wait minutes for your code to update, and while there is no way to improve the speed of this, you can check if the code you see on the site is the latest code that was last saved using the editor.

Whenever you save a test in Optimizely, the console displays the next revision number that it is saving to and repeats this until it has fully invalidated to the CDN.

revision-attempt

The first line that is highlighted displays the previous revision (“12”) along with the new revision (“13”) and the number of times it has been attempted (in this case it only attempted it once but this has been known to go over 100 if the Optimizely platform is under large load). The second line shows that it has completed updating the CDN to correct revision. Edit: As noted by Toby Urff from Optimizely, the editor also shows when the snippet is up-to-date by showing “(Uploading to CDN)” next to the “Save” button while it saves and then is hidden once the revision number of the snippet matches the revision number of that save.

save-cdn

save-cdn-complete

Once you have the test saved, you can go to your page and check the current revision by entering the following into the console:

window.optimizely.revision

This will return the revision number that you are currently in

revision

This, combined with checking the log, is incredibly useful for debugging and checking your code is running correctly and you are seeing the latest version. This should also speed up your development time as you know exactly when the code has been updated.

6. Use the page’s jQuery to access more functions

As I stated above, Optimizely provides a reduced snippet of 1.6.4 that doesn’t have some commonly used functions such as .hide(), .ajax(), .getScript(), etc. This can provide a problem when attempting to build larger more complex tests that may require external APIs/scripts. There are a few solutions to this: either load the full version of jQuery via Optimizely or not to load jQuery at all and use the page’s version.

Loading the full version via Optimizely can be a quick solution, but at the same time you are significantly increasing the snippet size. It also becomes redundant if you already have jQuery on the page. On the other hand, while using the page’s version of jQuery may seem at first to be a better option in terms of overall file size, it can actually be slower if you are loading jQuery at the end of the page (as it is generally recommended that you should do).

A middle ground to these two options is to include the reduced snippet with Optimizely, but use the website’s version to access functions such as .ajax(). To do this, you will need to write a function within Optimizely’s force parameters to check the existence of the page’s jQuery and then execute your script. This could look something like this:

/* _optimizely_evaluate=force */
function checkjQuery() {
    if (typeof window.$ !== "undefined") {
        window.$.ajax({
            url: '/path/to/file',
            type: 'POST',
            data: {param1: 'value1'},
        }).done(function() {
            console.log("success");
        });
    } else {
        setTimeout(checkjQuery, 50);
    }
}
checkjQuery();
/* _optimizely_evaluate=safe */

Here you can see that we are accessing the website’s version of jQuery using window.$ and making sure it is not ‘undefined’. We are then accessing the .ajax() function via the window.$ scope to access a fictional file called ‘/path/to/file’. This code should load as soon as jQuery is available on the page. One thing to be aware of when using this method: if you are adding or changing content in an element that you create outside of the force parameters, then you must check for the existence of that as well as the page’s jQuery. This is because there is a possibility of the page’s jQuery loading faster than Optimizely can get to the line of code that creates the new element.

In conclusion

These tips have been slowly accrued while I have been working on the Optimizely platform over two years as a developer for Conversion.com. These have proven invaluable to me and I hope they bring the same benefit to you as well.

Did you find anything useful in here? Do you have essential tips for coding in Optimizely that haven’t been mentioned? Get in touch with us in the comments and let us know your thoughts!