Web Interface Walk Through

Basics

Merge Launch is a graphical front-end to merge-based testbeds. It allows an experimenter to create, edit, and delete projects, experiments, realizations, and experiment models and interact generally with a testbed. It uses the MergeTB REST API to do this.

Note that Launch is a work-in-progress and may become unsynchronized occasionally. If it does, a refresh will generally resynchronize it.

Also note that the screenshots below were taken in a test environment and they show the site as not secure. The actual Launch site is only accessible via HTTPS and is secured with valid certifcates.

This page will demonstrate how to get started with Launch - from account creation through materialization a real experiment on testbed hardware. The basic steps are:

  1. Create and activate your account
  2. Create an experiment and push an experiment model
  3. Realize (reserve) resources and materialize (instantiate) the model
  4. Dematerialize and free realized resources

Account Creation and Authentication

There are three steps to having an active and usable account on the testbed.

  1. Create an account. The testbed uses the Merge OAuth2 system for creation and authentication of accounts.
  2. A testbed admin must activate the account. Until your account is activated by an admin, you will not be able to interact with the testbed.
  3. Upload an SSH formatted public key.

Account Creation

Start by opening a browser and navigating to https://launch.mergetb.net. Click the "login" link in the center panel or the login link in the upper right.

Initial View


This will take you to the Merge OAuth login, where you can create an account. Click the "Sign Up" tab, give a valid email-address, the username you'd like, and a password. Then click the "Sign Up" button at the bottom. This creates your account.

Create Account

After you've created the account, Auth0.com will send you an email to verify your MergeTB account. Please verify the account.


Activating the Account

Once the account is created, you'll be redirected to the initial Launch page. Note that you do not have access to any data and all pages will be empty as seen below. You'll know your account has been activated by the testbed adminstrators when a project name which matches your userid is seen in the "Your Testbed" view. This is your "personal project".

Inactivated

Click the Login link in the upper right and login, if you're not logged in already. If your account is not active (which yours is not at this point), Launch sends a message to the system telling it to initialize the account and create your personal project. This project will be your initial empty project.

The system does not automatically notify testbed OPs (yet), so please login to the #general channel of the DComp Slack and request OPs activate your testbed account.

Add Your Public Key

Once your account is activated, upload a public key to the portal. This key will be inserted into your Experiment Development Containers (XDCs) you create and your materialized experiment nodes. Click the User Profiles link in the header and click the + in the Public Keys widget.

NOTE: We only support SSH public key format keys at the moment. We do not support PKCS#1 PEM formatted keys. The key you upload must start with ssh-rsa ... and not -----BEGIN RSA PUBLIC KEY-----.

We recommend not using passphrase protected keys. (If you do use passphrase protected keys, you need to run exec ssh-agent bash and then ssh-add when using these keys in an active experiment.)

Add Public Key


Activated Account

Once your account is activated, you'll see your personal project.

Activated Account


Materialization Walk-through

To get an experiment materialized on the testbed, do the following:

  1. Create an experiment
  2. Load and push a model to your experiment
  3. Realize the model
  4. Accept the realization proffered
  5. Materialize the realization

Experiments are contained in projects. To create an experiment within a project, click the project in the "Your Testbed" view, then the "+" in the Experiments panel.

New Experiment


Fill out the experiment details, including the experiment name and description. Push the Create Experiment button to create the experiment.

Note that experiment names can only include lowercase alphanumeric characters. If you include uppercase characters, Launch will convert them to lower case before creating the experiment. In this case we create an experiment called "myfirstexp". Create Experiment


Once created, the experiment is shown in the list of experiments for the project. When you click on the experiment name, you are taken to the experiment view. Note there are no details about the experiment as it's empty at the moment.

Empty Experiment


To push an experiment model to the experiment, navigate to the Model Editor by clicking the Models link in the header. Then click the + in the Local Models table as shown here. This will spawn the Model Editor.

Create Experiment Model

From this dialog you can load an existing model by clicking Choose File or create a new model by not choosing a file. (If you don't load a file a basic model will be loaded for you.) In this case, I've loaded the lan3 model from our library of models at https://gitlab.com/mergetb/models.

Loading a Model

Loading a model will take you to the Model Editor as shown below.

Model Editor

From within the Model Editor view you can edit, visualize, and save (download) models. You can also push a model to a new or existing experiment.

To confirm the model will compile and generate a valid topology, push the Visualize button. When the button is pushed, Launch sends the model to the Merge Portal to be compiled and the topology for the model is returned. This is then displayed beneath the editor as shown here.

Visualized Experiment Model

Once you've confirmed the model is valid by visualizing, you push the model to an experiment for realization. Click the Push To Experiment link in the model editor header. This will spawn a pop-up dialog that lets you push the model.

Push Experiment Version

Choose the experiment you want to push the model to via the Project drop-down. (This drop down is populated with projects you have access to. Note that you only have access to your personal project at this point.) The Experiment drop-down is populated with experiments within the chosen project that you have access to. If you change the project, the list of available experiments will change. If you type in a non-existent experiment, a new experiment will be created. (The dialog will warn you when creating a new experiment.) In this case, we choose the experiment we created above, "myfirstexp".

Push Model

Once pushed, you will be returned to the editor. To see your new experiment model, navigate to the experiment page (Your Testbed, myfirstexp). You can now see there is an entry in the Experiment Versions list. From this entry you could edit the model more or view the existing topology of the experiment version.

Model Pushed


Realize Experiment Model

Choose the version of the model you wish to realize, in this case there is only one version. Click the + next to the version.

If you want to see the model, you can click the Topology of the model. This will show a basic visualization. If you want to edit the model (and possibly push a new version), you can push the edit link in the Model column.

New Experiment Realization


Choose a name for your realization and click Create. In this case we chose to name the realization "myfirstrlz".

Create Realization


After a short pause, the realization is created. Your experiment has reserved actual testbed resources for a short time. You may view the model, and if it meets your needs, you accept the realization.

Click the Accept button to reserve the testbed resources until you free them. These are now your resources until you free them.

Note that if you do not accept before the count down, your testbed resources will be automatically un-reserved and you'll need to re-realize the experiment model.

Accept Realization


Once accepted, the testbed resources are yours. Note the expires column is now Never. To materialize the realization, click the Materialize button. This will cause the testbed to image the nodes and create the experiment model on the testbed itself. When this is complete, the experiment is ready to be used.

Materialize


After the Materialize button is pushed, the materialization begins. Launch will poll the Merge Portal for the materialization status every two seconds until the materialization is complete. Here is an experiment being materialized. Note the list of materialization tasks are not complete. (Some are still in a False completion state).

Active Materialization

After a few minutes all materialization tasks are complete and the experiment is active.

Active Materialization


Connecting to experiment nodes

At this point, the experiment is active and the nodes are imaged and the topology specified by you is configured.

You connect to your experiment nodes through an Experiment Development Container (XDC). You spawn this container as needed to interact with your experiment. The XDC comes installed with the mergexp library for experiment design, mergetb, the command line tool for interacting with a Merge testbed, and a Jupyter Notebook interface (and its web-based terminal). The base XDC image is Ubuntu 18 and has internet access for apt-installing any needed software. Persistent per-project directories and persistent user home directories are mounted on each XDC.

  1. Create an XDC in an experiment and login.
  2. Attach to the experiment materialization. This sets up DNS and a secure tunnel to the experiment gateway.
  3. Use the nodes via SSH.
  4. Detach from the experiment.
  5. Optionally delete the XDC.

To create an XDC in an experiment, navigate to the experiment page (Your Testbed, click on experiment name). Click the + at the bottom of the Experiment Development Containers widget.

Spawn an XDC

Clicking this leads you to a spawn XDC dialog. Choose a name for your XDC and click Spawn XDC.

The new XDC is added to the list of XDCs for this experiment. The Jupiter Notebook field will show a loading visual until the XDC has been spawned in the portal. Once spawned, the entry will become a link to the Jupyter Notebook running on the XDC.

XDC Created

Once the XDC is created, you can login to it then from it connect to your experiment's nodes.


There are two ways to login to the XDC, ssh to it through the Merge jumpstation, or directly via the Jupyter terminal. (Note to ssh directly you must have uploaded your public keys to Merge.)

To connect via the web-terminal, click the Jupyter Notebook link for the XDC you wish to connect to. This will spawn the Jupyter Notebook interface to the XDC. From that, click New and choose Terminal.

XDC Terminal

This will open a web-based terminal session in your XDC. It logs you in as root, but you can su - [user] to login as yourself. Note the project and user persistent directories in the screen shot below.

XDC Logged In


To connect to the XDC via ssh, use the Merge jumpstation node and ssh through it:

$ ssh -J jumpc.mergetb.io:2202 xdc-testexp-gtl

The name of the jumpstation is jumpc.mergetb.io and it listens for ssh connections on port 2202. The XDC reference name is of the form [xdc name]-[experiment name]-[project name]. For example assuming a project dcomp, an experiment named test, and an XDC named johnboy, the XDC reference name is johnboy-test-dcomp. The reference name for an XDC is shown in the Reference Name column of the XDC widget in the experiment view in Launch. (In the above image there is one existing XDC with reference name myfirstxdc-myfirstexp-maisie.)

The first time you connect to an XDC, you may want to copy the private key (that pairs to the public key you uploaded via the GUI) to the XDC. This will allow SSH connections from all XDCs to your experiment nodes to work without any more configuration. The system places the public keys that are uploaded unto your experiment nodes as they are materialized. Thus if the private key is on an XDC (usually at ~/.ssh/id_rsa), SSH connections between the two will just work. Your home directory on XDCs is mounted when the XDC is spawned, thus it exists independent of the lifetime of any one XDC.

We also have some tips for using ssh with your XDCs here. Please take a moment to read through this.


Once logged into an XDC, you access your experiment nodes by attaching the XDC to the materialization. This creates a secure tunnel between the XDC and your experiment gateway and sets up DNS to resolve the names of your experiment nodes. To attach, you use the xdc utlity tool in /usr/local/bin/xdc. To create the tunnel, the xdc utility calls out to the Merge API. And to you use the Merge API, you must be authenticated. To authenticate, run xdc login <username>. The username and password is the same one you've used to login to the GUI. If you're not logged in xdc will exit reminding you to login. The arguments to xdc attach are project name, experiment name, and realization name.

glawler@one:/tmp$ xdc attach -h
Build a tunnel between this XDC and the given materialization.
Usage:
xdc attach <pid> <eid> <rid> [flags]
Flags:
-h, --help help for attach

Once logged in, you will remain logged in until you log out.

Here is a sample session. In this session the project is glawler, the experiment is hi, and the realization is one.

glawler@one:/tmp$ xdc show tunnel
xdc not attached: no wireguard interface found
glawler@one:/tmp$ xdc attach glawler hi one
glawler@one:/tmp$ xdc show tunnel
Materialization: glawler hi one
Interface: wgdtwhq (UP)
Address: 192.168.254.12/32
glawler@one:/tmp$ ping n00
PING n00.one.hi.glawler (172.30.0.10) 56(84) bytes of data.
64 bytes from 172.30.0.10 (172.30.0.10): icmp_seq=1 ttl=63 time=3.90 ms
64 bytes from 172.30.0.10 (172.30.0.10): icmp_seq=2 ttl=63 time=3.05 ms
^C
--- n00.one.hi.glawler ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 3.052/3.476/3.900/0.424 ms
glawler@one:/tmp$

Once attached, you can confirm the connectivity to all experiment nodes by running xdc ansible ping. This will use ansible to ping each node in your experiment and report the results.

glawler@one:~$ xdc ansible ping
n01 | SUCCESS => {
"changed": false,
"ping": "pong"
}
n00 | SUCCESS => {
"changed": false,
"ping": "pong"
}
n03 | SUCCESS => {
"changed": false,
"ping": "pong"
}
n04 | SUCCESS => {
"changed": false,
"ping": "pong"
}
n02 | SUCCESS => {
"changed": false,
"ping": "pong"
}
n05 | SUCCESS => {
"changed": false,
"ping": "pong"
}
glawler@one:~$

The xdc command can also generate ansible inventory and etc-host style files for your attached experiment. See xdc -h for usage.


A per-project persistent directory is created for each project. This project is mounted at /project on each XDC. There is also per-user persistent storage mounted as your home directory mounted under /home. The lifetime of these directories are independent of XDCs. They are created with each new project and each new user.


Connection Cleanup

When you're through working with your experiment you may (optionally) detach from the experiment nodes and (optionally) destroy the XDC.

To detach, run the xdc detach command from within an XDC.

glawler@one:~$ xdc show tunnel
Materialization: glawler hi one
Interface: wgdtwhq (UP)
Address: 192.168.254.12/32
glawler@one:~$ xdc detach
glawler@one:~$ xdc show tunnel
xdc not attached: no wireguard interface found
glawler@one:~$

To delete the XDC, navigate to the experiment view in Launch (Your Testbed and click the experiment name) and click the x at the end of the row for your XDC in the Experiment Development Containers widget.

Note that deleting the XDC will not destroy per-project or per-user storage.


Experiment Cleanup

To dematerialize the realization, push the Dematerialize button.

Note that this will not free the resources of the realization. Those testbed resources are still reserved for you. To free the resources, you must delete your realization.

Dematerialize


To delete the realization (after you've dematerialized), navigate back the experiment view, and click the x next to the realization you wish to delete. Once done, the testbed resources are freed for other to use. You may of course, re-realize to re-reserve.

Delete Realization

Note that you can also delete a realization by deleting the experiment that the realization exists in. In general when you delete an entity, resources under that entity are deleted. If you delete a realization, materializations of that realization are deleted. If you delete an experiment, its realizations (and thus any materializations) are deleted. Etc.


Adding Project Members

To add a member to your project, navigate to the project view of the project and click the + in the project members list.

Note the username must match an existing user. Once the user is added to the project, they will have access to everything in that project. Their username will be added to the project's group and thus they will also have access to the shared project folders.

To remove a project member, click the x at the end of the member's row in the list.

Add Project Member


Added members will have read and write access to the project and they will be assigned a member role. Note that only the creator of a project can delete the project.

Project Member Added