CoCalc now offers an option to render LaTeX using KaTeX rather than MathJax. At the moment, KaTeX is an experimental feature which is turned off by default. To enable it, open Account / Preferences, and under Other Settings, check the box next to “KaTeX: render using KaTeX when possible, instead of MathJax”.

enabling KaTeX in Account Preferences

KaTeX is often over 100 times faster than MathJax, but it doesn’t handle all expressions covered by MathJax (or LaTeX). In these cases, CoCalc with KaTeX enabled will still fall back to MathJax. The selection happens for individual expressions, so one expression in a markdown file or a notebook cell might be rendered with KaTeX, while another would be rendered with MathJax.

About the author: Alex Staravoitau is a software engineer and machine learning enthusiast, the creator and main contributor of Juno, which is a Jupyter Notebook client for iOS.

Juno

At Juno, we all have been huge fans of Jupyter for awhile, and most importantly of the flexibility it offers: we strongly believe that the fact that you only need a screen and network connection to get access to pretty much unlimited computational resources has enormous potential. Naturally, we thought that Jupyter could use a proper client iOS application with a native interface, that would let you connect to a remote backend and work with Jupyter on your iPad. Now, after months of making and beta testing our app, Juno has made it to the AppStore!

screen capture of Juno running on iPad

Get a Running Start on the Learning Curve

The CoCalc Library is a great way to get up to speed with new material.

Getting started with a new programming language or software package can be a slow process. It takes time to find your way around new terminology, development workflow, and error messages. Reading documentation and tutorials is helpful, but for most of us, the process of learning speeds up dramatically when we start writing code and actively experimenting.

The CoCalc Library makes it easier to get started with a new toolset or a new topic. It gives you access to a curated collection of open source documents and code from educators and software developers. With a single click, copy a selected topic into a CoCalc project. Then you can review, modify, and run files as desired.

Using the Library

Here’s an example. Suppose you want to start working with linear algebra in Sage.

1. Open the Files view in the directory where you want example documents to be placed. In the screenshot, that directory is ‘EXAMPLES’.

2. Click (+)New, and scroll to the middle of the browser tab to see the section with a label of “Library” on the left.

Minerva Schools at KGI is an innovative university that makes extensive use of web technology to reinvent the classroom experience. CoCalc is collaborative web-based software that makes it possible to very easily use the full suite of modern open source technical software, including Jupyter notebooks, LaTeX, Sage, and R. We launched CoCalc in 2013, originally under the name SageMathCloud.

Minerva approached us last year because they needed to embed collaborative Jupyter Notebooks in their platform to support teaching computer science and other data intensive courses. Throughout much of 2017, we supported Minerva’s integration of CoCalc with their Active Learning Forum, and today they are one of our biggest customers.

Last year we widened our focus and renamed the project from “SageMathCloud” to “CoCalc”. You can work with SageMath, Python, R and several other languages. The focus is:

Collaborative Calculation in the Cloud

This means we designed our service with collaboration in mind. Changes to an interactive document are synchronized with all collaborators of the project. They are invited by you and can be your colleagues, friends or in some cases, your students.

Python, SageMath, Octave, Julia or R?

CoCalc speaks many of the best open-source languages and brings along a large set of associated libraries. This means you no longer have to setup your own software environment and continue to worry about updating and maintaining it.

Have a look at the list of currently installed libraries for Python, R Statistical Software, Julia, and a selected list of available executables.

Native Jupyter Notebooks

CoCalc offers its own unique integration of Jupyter Notebooks. You can start by uploading your files or creating a new file with ease. There are many kernels available and a large collections of libraries are regularly updated. All cells and the state of the kernel are synchronized in real-time with your collaborators. Share ideas in a side-by-side chat and review changes via time-travel history. Once you’re done, you can download your worksheet as a standard *.ipynb file or publish the notebook online via CoCalc’s own share server.

LaTeX Editor, Sage Worksheets, chat, teaching courses, and much more

All previously available applications are still fully supported. In fact, we spent endless hours in debugging a lot of issues, improving the overall performance, and invested heavily in the overall system stability. We switched to a fully containerized system using Docker and Kubernetes helps us to take care of operational details.

Below is an assorted collection of screenshots to show you an overview. You can edit LaTeX documents in a side-by-side editor, do mathematics in Sage Worksheets, access a wide range of software in the underlying Linux terminal, and even teach a course using our advanced course management feature.

Publishing documents

And finally, after you have finished your work and you want to share it with the world, CoCalc offers you the ability to publish selected documents online.

Under the hood, CoCalc provides you with a very flexible and versatile Linux environment. Due to the upgrade to Kubernetes, our (unofficial) support for using crontab files for periodic tasks was removed. Instead, there is a new and more flexible mechanism to use regular Bash, Python, or SageMath files to accomplish even more.

Project Initialization

When a CoCalc project starts up, an instance of Supervisor is started and responsible for running:

• an instance of “local hub”. It is used for managing the project, communication with the outside world, monitoring, etc.
• sshd: the endpoint for remote SSH access
• initialization file: it sit inside your project and is only started if it exists. It is located in your home directory and named project_init.sh. As the name suggests, it is run via the Bash interpreter.

Initialization file management

Supervisor is configured to run this initialization file like background processes. This means it checks the exit value of the process and restarts it if this code is not 0 or 2. Therefore, a script that runs without an error is not restarted.

On the other hand, if the process is terminated, interrupted or crashes – therefore the exit code is e.g. 1 – the process is restarted. This adds some robustness to long running background jobs.

Example 1: record start time

A very simple example is to record the project’s start time. Go to your home directory and create a file project_init.sh with that content:

date > project-start

This is a very simple bash script, which pipes the output of the date command into the file project-start.

In order to see its effect, the file needs to be saved and the project restarted. Give it a few seconds to come back online and run the script. After that (maybe click the refresh button in the file listing) you should see this file and its content might look like:

Mon Sep 12 11:14:20 UTC 2017

Other languages besides Bash?

You can run any language via bash’s exec! For example, project_init.sh containing

exec python3 project_init.py

will run a Python 3 initialization file named project_init.py.

CoCalc support for SSH is significantly improved with the CoCalc upgrade to Kubernetes. This article reviews new features and gives examples using SSH with CoCalc.

New SSH Features

TL;DR: SSH to CoCalc projects has been completely rewritten. For example, you can now paste a single ssh key in account settings, and it will automatically be available in all of your projects!

User Interface for Managing Public Keys

User interfaces are now available for managing your SSH keys. Using these forms, you can add and remove keys, as well as viewing which keys are configured and when they have been used last. There are separate UI dialogues for keys configured at account and project scope, as described below.

To connect from a remote computer to the SSH server on a project, you must have owner or collaborator status on that project. It is NOT required that the project have network access enabled; network access is only required if you want to ssh from a project to somewhere else.

NOTE: Manual editing of the authorized_keys file is no longer supported. Use the procedures below to add public keys.

Configure a single key for all projects for a given account

It is now possible to set a public key for your CoCalc account that works with all projects where your account is owner or collaborator. Click on your account name or the gear icon next to it, and choose the SSH Keys tab. There is a form at right for entering keys, as shown here:

To add a key, enter a descriptive title and paste the .pub file for the public key file into the form and click Add SSH Key. Extraneous spaces and newlines in the key that is pasted in will be automatically removed.

The title, MD5 fingerprint, and last-used timestamp will be displayed for each key that is added, as shown here:

Multiple keys can be added in this way.

If you want to remove an account key, return to the SSH Keys tab and click the Delete button to the right of the key you want to delete.

To use an account key, you will need the ssh user and hostname for a project you want to access. Open project settings (wrench icon) and look in the SSH Keys section at lower left for the information under ‘Use the following username@host:’. You will see that the username is the same as the destination project id without hyphens, and the host is always ssh.cocalc.com. (This is a major improvement over before when the host depended on the project.)

See below for detailed examples of using SSH keys.

Configure a public key for a single project

There is also a form to add and remove keys for a single project. To add a key for use with a single project, open project Settings (wrench icon) and scroll down to the SSH Keys section at lower left. Click ‘Add an SSH Key’ to open the dialogue shown below.

SSH agent forwarding

If you’re familiar with ssh-agent forwarding, this feature is now supported by CoCalc. You will need to have ssh-agent configured on your local system and have public keys distributed to all target hosts for agent forwarding to work.

See Steve Friedl’s Tech Tips guide for more info about SSH agent forwarding.

SSH access between CoCalc projects

When the client and server for an SSH connection are both CoCalc projects, you can use a hostname of ssh rather than ssh.cocalc.com. Using the shorter hostname gives a much faster connection, because it is optimized within the CoCalc cloud.

As of May 20, 2017, SageMathCloud is being renamed to CoCalc, for Collaborative Calculation in the Cloud.

A Natural Evolution

SageMathCloud was introduced in April 2013 as a hosted platform for the SageMath software system. It saved thousands of users the time and effort of building, installing, and updating SageMath locally. SageMathCloud provided a single consistent platform for improving the range and quality of services offered.

Over time, we began adding new features, mostly for our own needs: a course manager to teach students SageMath, chat to coordinate within the platform, a $\LaTeX$ editor to write documents, and Jupyter notebooks to support fellow instructors. In addition to conventional backups, we added snapshots and time travel. We added support for numerous languages, including Python 3 as well as Python 2, R (including R markdown), the complete Anaconda distribution, Octave, Julia, C, C++, Haskell, and Scala. We included major libraries and packages for these languages, and update them regularly.

We became a platform that enables students, teachers, and researchers to collaborate using a broad selection of programming languages and development tools – not just SageMath.

While SageMathCloud suggests exactly what we started out as, we have certainly outgrown it. The new name, CoCalc, reflects the evolution into a general purpose, cloud-based calculation and computation platform. Our recent move to rewrite the Jupyter notebook to be more integrated and fully supported is an example of how our mentality has shifted.

The CoCalc Experience

Newcomers can explore CoCalc services from different approaches:

• Projects. In CoCalc, work happens inside projects. A project is a personal workspace where you can group related files.

• Collaboration. By inviting collaboraters into your project, you allow them to see and chat about your work, and even edit it with you in real time.

• Teaching. With Cocalc’s teaching tools, you can setup courses, create handouts with mathematical formulas, and manage homework and grading.

• Authoring. CoCalc has extensive support for $\LaTeX$, in document editing as well as all interactive notebooks and chat.

What Should I Do If I’m Already an SMC Subscriber?

When you visit https://cloud.sagemath.com after the change, you’ll be immediately redirected to https://cocalc.com. You’ll then have to sign in again the first time. That will set a cookie, and after that everything will be as normal. Anyone following a link into the old name, e.g. https://cloud.sagemath.com/some_path..., will be redirected to the same path under the new name, https://cocalc.com/some_path....

You will notice our documentation changing to use the CoCalc product name and the CoCalc.com domain name.

We have created a new CoCalc discussion group in Google Groups here and we will automatically subscribe all current sage-cloud subscribers to it.

What Stays the Same

The company behind CoCalc will remain SageMath, Inc. Our staff email addresses will remain in the sagemath.com domain.

More importantly, we remain committed as ever to improving the experience of scientific and exploratory computing for students, teachers, and researchers around the world.

Try CoCalc Today

If you’re not already a subscriber and wonder what all the fuss is about, now is a great time to get acquainted. Start CoCalc for free today.

Today SageMath, Inc. is pleased to announce the release of a completely new Jupyter notebook implementation, optimized for SageMathCloud, our cloud-based, collaborative environment.

This new implementation was motivated by the need for better collaboration and browser/server synchronization. Rewriting also offered an opportunity to integrate previously-unavailable features of SMC into Jupyter notebooks. However, throughout this process we minimized interface discrepancies between old and new versions so that switching is as seamless as possible.

Here’s an overview of the most significant enhancements in today’s release:

Collaboration

Simultaneous Editors

Simultaneous editing by multiple people is now fully supported, including multiple cursors and document-wide, user-specific undo and redo.

User One and User Two editing a notebook at the same time.

Chat

Each notebook now has a chat sidebar. As always, chat supports markdown, $\LaTeX$, and video chat.

Text chat with markdown shown in panel to right of notebook body.

Server-side enhancements

TimeTravel

Jupyter includes TimeTravel just like our other editors. It shows detailed history of all changes to a notebook, and the author of each change.

Time Travel view of previous notebook as seen by User Two
showing revision 93 (of 95) created by User One.

Output

• Sophisticated handling of large output: throttling, windowing, back-end buffering.
• Background capture of execution output, even if no user has the notebook open in their browser.

User interface

• Cleaner, more modern look with buttons and menus that better reflect state.
• Better mobile web support.
• Uniform font sizing.
• Code folding.
• Vim, Emacs, and Sublime keybindings, and color schemes (in account settings).
• Export notebook file to LaTeX using “File > Download as… > LaTeX (.tex)…”.

Sample notebook about to be exported.

Exporting notebook to Latex and opening the .tex file in SMC.

Converted ipynb file opened in LaTeX editor.

• A purely client-side notebook viewer for easily sharing your work publicly.
• Raw file edit mode: synchronized editing of underlying JSON ipynb file. See resulting changes to the rendered notebook in real time.

Introduction

Initially motivated by the shutdown of the RethinkDB company, and the licensing situation with RethinkDB (a blocker for certain parts of my business), I worked very hard for two months to completely rewrite the realtime and database components of SageMathCloud (SMC) to use PostgreSQL instead of RethinkDB, initially motivated by this discussion in Hacker news. I battled with and used RethinkDB heavily since May 2015, and I’ve used PostgreSQL heavily as well, with production data, rewriting all the same queries in both systems, so I’m in a good position to compare them for my use case (the site SMC).

This is my story. It’s a personal comparison, with NO BENCHMARKS or hard data you could reproduce. It’s what I would tell you if we were talking by the water cooler.

Summary:

• I’m very happy with the rewrite.
• Everything is an order of magnitude more efficient using PostgreSQL than it was with RethinkDB.
• It is much easier to do exploratory queries of our data using PostgreSQL than it was with RethinkDB. PostgreSQL is much more expressive than ReQL, has a massive number of built-in functions, so we are making much better use of our data. With RethinkDB, often we just ended up greping through the latest database dump.
• PostgreSQL is “statically typed”, whereas RethinkDB had no type or schema enforcement at all; explicit clear typing improved the quality and robustness of our application.
• We are saving $800 month (!), due to reduced CPU and disk space requirements.
• I had no clue that RethinkDB would be Apache licensed in February 2017.