Body
Quest OnDemand
Quest OnDemand is a web portal that provides a single, user-friendly access point for Quest, Northwestern's High-Performance Computing (HPC) Cluster. It allows users to submit batch jobs and manage, view, edit, and create files. In addition, it enables users to run Graphical User Interfaces (or GUIs) like Matlab and Integrated Development Environments (IDEs) like RStudio Server and Jupyter seamlessly. Knowledge of the command line or Linux is not needed to effectively utilized Quest OnDemand. It is the ideal way to access Quest for new users or those who would like to use graphical applications on Quest.
In addition to this User Guide, we encourage folks to watch our whole videos series on Using Quest OnDemand.
Logging into Quest OnDemand
To learn more about logging into and navigating the Quest OnDemand web portal, please watch our video: Quest OnDemand: Logging into and Navigating the Web Portal.
To connect to Quest Open OnDemand, please first ensure that you are connected to the on-campus WiFi (eduroam) or Global Protect VPN.
Once connected as directed above, direct your browser to https://qondemand.ci.northwestern.edu which will trigger signing in via Northwestern's Single Sign-On (WebSSO). After you have authenticated via WebSSO, you will see a landing page with a dashboard. The navigation bar will look something like below.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=7ef2cace-c8cc-4261-b7b4-3986f180a500.png&beidInt=2)
File Explorer and Navigating Quest
To learn more about how to use the Files applications to manage your files on Quest, please watch our video: Quest OnDemand: File Management and Upload.
You can access the Quest file system through the "Files" tab. Under "Files", you will find shortcuts to your HOME directory, allocations directories, and scratch space, if you have one. Files can be viewed, edited, renamed, deleted, and created. Files can also be moved and/or copied between folders on Quest. Files can be uploaded to Quest from your computer using the upload button, and from Quest to your computer with the download button. When transferring large files or large sets of files we recommend using Globus.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=a1185db7-1049-4b1f-938c-ad0c04a28565.png&beidInt=2)
Job Composer and Submitting Jobs
To learn more about how to use the Job Composer application to create and submit jobs on Quest, please watch our video Quest OnDemand: Using the Job Composer Application to Create and Submit Jobs on Quest.
Jobs can be monitored, created, edited, and scheduled using the tools under the Jobs menu. The Jobs navigation bar has two options, Job Composer and Active Jobs, which we describe in detail below.
Job Composer
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=f55f2f52-4582-4799-80d1-991760b42d81.png&beidInt=2)
The job composer is a GUI alternative to the Slurm job scheduler with detailed information about what each option is when submitting a job while still acting with the same capabilities a scheduler has. There are three ways to create and submit jobs to Quest using the job composer application. We highlight each of these options, "From Template", "From Specified Path", and "From Selected Job", below.
From Template
To start the process of creating a new job
From Template, select
New Jobs ==>
From Template
will allow you to make a start a new job submission process using either an example job submission template that have been provided by Northwestern IT Research Computing and Data Services team or that you have created yourself (more on that later). The current example submission script options available include Python, R, Matlab, and VASP, but we will continue to add more template over time.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=17c69968-735e-4204-b708-17219850712f.png&beidInt=2)
Selecting one of the templates will cause a preview window to show on the right hand side of the screen. It will tell you what the name of the job is, what the name of the submission script is, and whatever other files are needed to successfully run this job. To start the processing of submitting a job, you will click
Create New Job.
After doing this, it will be added to you Job Composer queue where it is important to note to things. First, creating this job and adding it to your queue did not submit the job.
Second, the job submission script still needs to be filled out with details that are specific to you and your job, such as the partition, the allocation, the walltime, etc. For more information on setting the values of a Slurm job submission script, please see our
Everything You Need to Know about Using Slurm on Quest KB Article. To do this, you can click the "Open Editor" button in the bottom left hand corner of the job submission preview and it will allow you to edit the file in your browser.
Once you have successfully edited this job submission script with the details specific to you, you can now submit the job by selecting the green submit button which will submit the job and provide it with a Slurm job ID and and job status.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=80089967-9331-4838-b1df-bd0e582801e4.png&beidInt=2)
From Specified Path
If you have a folder containing a job submission routine on Quest already and would like to import it into the Job Composer interface so that you can submit and manage the job from there, then
From Specified Path option is perfect for you.
To start the process of creating a new job From Specified Path, select New Jobs ==> From Specified Path, which will take you to a page with the following web form:
You must fill this form with the following information in order to import this existing Quest workflow/job into the Job Composer interface.
- The full path to the folder containing the submission script and any files needed to run the job
- A name for the job which will determine the Job Name as it appears in the Job Composer interface.
- The name (i.e. filename) of the submission script itself.
- The cluster name: Always Quest S
Below you can find an example of what filling out this form might look like. After filling out the form, click Save and the job will be added to the Job Composer interface.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=c69eb41c-6e93-4bac-b5c7-f62c1665f16d.png&beidInt=2)
From Selected Job
The
From Selected Job options is great if you want to copy/replicate a previous job that is already in your list of jobs in the Job Composer queue and make minor changes.
To start the process of creating a new job
From Selected Job, select
New Jobs ==>
From Selected Job.
You should then see a screen which indicdates that this job has been duplicated in the Job Composer interface successfully.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=80764a10-cc19-4693-8bea-72fb9ef67f7a.png&beidInt=2)
Active Jobs
Active jobs can be viewed within the Active Jobs tab that provides a list of active jobs with information on user, account, time used, queue, status, cluster, and actions for each of the jobs running. Running and pending jobs can be cancelled from the active jobs list by clicking delete.
Interactive Applications
Quest OnDemand interactive applications allow users to run IDEs and GUIs including MATLAB, Abaqus, Lumerical, Jupyter, Mathematica, RStudio server, GNOME desktop, Stata, and others on Quest compute resources. Running these applications can be challenging using traditional methods such as SSH tunnelling or X11 forwarding with FastX, but Quest OnDemand simplifies this process significantly. If there are any interactive applications that you would like to have added please contact quest-help@northwestern.edu.
Accessing Available Interactive Applications
Below we describe two different ways to access the available interactive applications on Quest OnDemand.
Accessing Interactive Apps From the Landing Page
Several interactive applications can be access directly from the landing page by selecting the corresponding icon. For instance, see an example of selecting the Matlab Interactive App below.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=bbc17343-26ac-483c-82f0-c7cca985a195.png&beidInt=2)
Accessing Interactive Apps from "My Interactive Sessions"
You can select the "My Interactive Sessions" tab from the navigation bar, as shown below.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=aa5994d2-a79b-47f1-a9b2-1d982197619e.png&beidInt=2)
This will take you to a page which both has information about any active sessions that you have and has all available interactive applications listed on the left hand side as shown below.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=f41f856f-b12e-4fea-9e9c-fd91c56ab213.png&beidInt=2)
After selecting an interactive application (in this case Mathematica), you will be presented with a web form asking you question that will be very familiar to those who have submitted batch or interactive jobs on Quest before.
![](https://services.northwestern.edu/TDPortal/Images/Viewer?fileName=8bb4d010-54ed-40a5-b596-759801108f01.png&beidInt=2)
Understanding Interactive App Form Options
Below, we discuss the shared options across the different interactive applications. We encourage you to read the help information on the options that are exclusive to a particular interactive application or to watch our How-To videos for the interactive applications Jupyter, RStudio Server, and Matlab which are linked at the end of this User Guide.
Partition
Select the SLURM partition you will submit this job to. This will set the value of the -p
or --partition
flag for your job. This selection will impact a number of other options on this form, including which accounts/allocations and maximum walltimes you can choose and whether or not you have the option to select GPUs. For more information on partitions, please see the partitions section of our Everything You Need to Know about Using Slurm on Quest page.
Optional and Hidden Unless Partiton is Valid: GPU(s) that you would like to request
The number and type of GPUs you are requesting for this job, only available for certain partitions. This will set the value of the --gres
flag for your job. More information on GPUs can be found on the GPUs on Quest page.
Account
The Quest allocation/account under which you will be submitting this job. This will set the value of the -A
or --account
flag for your job. Please note that if you do not see an allocation option you expect, this means one of the following: that it is a storage only allocation, that your allocation is expired, in which case please request to renew it using the appropriate form that can be found here, or that the partition that you have selected cannot be submitted to from that allocation. More information on this parameter can be found in the account section of our Everything You Need to Know about Using Slurm on Quest page.
Walltime
Select the maximum number of hours you want this session/job to last. You can always terminate your session/job early, but it will automatically be terminated by Slurm once the walltime is hit. This will set the value of the --time
flag for your job. Your available options for walltime are effected by the partition that you have selected. For more information on walltimes, please see the walltime section of our Everything You Need to Know about Using Slurm on Quest page.
Number of CPUs
The number of CPUs you want for this session. This will set the value of the --ntasks-per-node
flag for your job. Please note that you likely do not need to request more than a single CPU unless you know that your application can be parallelized. Also, please note that the more cores requested, the longer the wait will be for the session to start. For more information on setting the number of cores, please see the number of cores section of our Everything You Need to Know about Using Slurm on Quest page.
Memory
How much total memory/RAM (in GBs) you want for this session. This will set the value of the --mem
flag for your job. Knowing ahead of time how much memory you need can be difficult. That said, you can use the utility seff
to profile completed jobs on Slurm which give the maximum memory used by that job. For more information on setting memory, please see the memory section of our Everything You Need to Know about Using Slurm on Quest page.
Optional Checkbox: Request More Than One Node
Only check this box if you are confident that your application of choice can utilize CPUs across more than one node. In Python, some of these applications include MPI4py, Dask, and PySpark. Once checked, you will be shown another field where you can request multiple nodes.
Optional: Number of Nodes
If you select the option above, then this option will appear and ask you how many nodes you want for this session. This will set the value of the -N/--nodes
flag for your job. A reminder to check that your application of choice can utilize CPUs across more than one node. These applications/packages would include MPI4py, Dask, PySpark, etc. For more information on setting the number of nodes, please see the number of nodes section of our Everything You Need to Know about Using Slurm on Quest page.
Optional: E-mail Address
Enter your email address if you would like to receive an email when the session starts. Leave blank for no email.
Jupyter Notebook or Lab
To learn more, please watch our video: Quest OnDemand: Running the Jupyter Lab or Notebook Interactive App.
RStudio Server
To learn more, please watch our video: Quest OnDemand: Running the RStudio Server Interactive App.
Matlab
To learn more, please watch our video: Quest OnDemand: Running the MATLAB Interactive App.