Document ToC

Introducing Pleiades

Concepts and Principles

History

Principle of the Pleiades Platform (MEC-RM)

Acknowledgement

Swagger UI


Pleiades for Business

TL; DR

Pleiades is designed as a distributed serverless computing platform.

(Work in Progress) Distributed means the pleiades platform itself can be distributed to several different reagons and be distributed to several instances inside each reagon.

Introduction

When your computer is not enough for you computing task, have you considered about rent some one else’s computer?

This idea itself is not so novelty. Maybe you have already heared the SETI@home project with the BOINC platform. SETI stand for "Search for Extra-Terrestrial Intelligence". the Arecibo Telescope in Puerto Rico will record their observational data on their hard disks, which were then sent to the Berkeley by bostal mail. The Berkeley will cut them to work units in approximately 0.35 megabytes or around 107 seconds of the recorded data. The work units will be distributed to the volunteer's computers and the results will be collected back over the internet. Before the closure of Arecibo Observatory, this project have connect around 145,000 active computers (1.4 million in total), and archive the ability to compute over 668 teraFLOPS. The SETI@Home project also release their computing platform as the BOINC project. As a high-performance volunteer computing platform, BOINC brings together 34,236 active participants employing 136,341 active computers (hosts) worldwide, processing daily on average 20.164 PetaFLOPS as of 16 November 2021 (it would be the 21st largest processing capability in the world compared with an individual supercomputer). World Community Grid is one of the projects using the BOINC platform released by the SETI@home project. It became famous by one of their subprojects: Open Pandemics - COVID-19. The Japan government also launched some grid computing projects such as the National Research Grid Initiative (超高速コンピュータ網形成プロジェクト). Similar ideas are also presented on some work of arts. For example, the Level Upper (Japanese: 幻想御手(レベルアッパー)) and the Misaka Network (Japanese: ミサカネットワーク) in the story of "A Certain Scientific Railgun" (Japanese: とある科学の超電磁砲) are two networks that connect each others to get more powerful. Also, the Autonomous Intelligence Fixed Artillery (Japanese: 自立思考固定砲台) or Ritsu (Japanese: 律) in Assassination Classroom (Japanese: 暗殺教室) shows the possiblity of connecting a large amounts of computers for her computing tasks.

This project have a similar object, although have a different history from other grid computing projects. We are tring to make the Pleiades project to be a grid computing platform with a similar interface like the Slurm workload manager, with a intergrated serverless computing support. Sure, this design have some historical reasons, You can see our history from History page. But,

Basic Designs

You can find a more detail information from the Concepts and Principles page.

现有的网格计算平台在设计之初通常都是完成某一特定计算任务,随着不断有新的计算任务加入而主逐渐发展壮大。这种开发思想是个人比较认同开发思想:使用计算机来解决某一问题,而非将使用计算机作为最终目的。在没有明确的目的的前提下开发设计程序可能难以避免过度设计的陷阱。但作为计算机基础设施研究项目,此项目无法避免地在毫无明确目标的前提下进行设计开发。这就要求我们在设计开发此项目时慎重考虑每一项功能的实用性与必要性。换言之,我们设计此项目时的一个重要思想是:Less is more。但是Less并不意味着残缺,我们仍然需要考虑那些功能是必要的。而这个问题就不可避免地从要求我们从“计算”与“交易”这两核心问题开始思考。

计算模型

在数学中,函数被定义为一种从输出到输出的映射关系。Lambda演算借鉴了这种关系。基于这种模型,我们可以得到一个高度抽象化的计算概念:我们将“计算”定义为一种“函数”($\lambda$)。此函数接受一个参数($P$),并且给出一个返回值($R$)。进一步,我们可以将由此三项参数唯一确定的一个计算过程定义为一个工作单元$J \coloneqq (\lambda, P, R)$。由此我们就得到了此系统中最为重要的全部四个概念:工作单元,函数,输入(参数)与输出(返回值)。

image.png

我们可以观察到,本系统无需关注输出和输出的具体内容。因此这两种数据可以被抽象为同一种类型的数据,这里我们将其定义为D。于是我们可以得到$J \coloneqq (\lambda, D_{input}, D_{output})$。至此,系统中所需的数据类型已经从四类缩减到三类:工作单元J,函数Lambda,输入输出D.进一步去思考我们可以如何定义Lambda。在冯诺依曼架构中,程序和数据并不存在本质区别。也就是说,Lambda函数亦可以与输入输出一样,被抽象成一个单纯的“数据”。也就是说我们似乎可以简单地定义$J \coloneqq (D_{\lambda}, D_{input}, D_{output})$。那么,代价是什么呢?

那么代价是什么呢?当我们放眼整个网络,我们会发现世界上存在各种各样的“计算机”。包括但不限于诸如AMD64架构或RISCV架构的物理计算机,执行JVM命令或WebAssembly的虚拟计算机。我们很难做到让同一份二进制数据

算力交易模型

既然本项目是一个算力交易平台,那么我们必然要对“算力交易“这一抽象过程进行建模。“交易”离不开买方与卖方,而在另一些场合,交易双方也许会被称为生产者和消费者。而在我们这个平台中,我们选择将使用计算力的一方称为依赖人(Requester),而将提供计算力的一方称为工作员(Worker)。而沟通Requester与Worker的桥梁则为上一节提到的工作单元Job。在最简单的场合下,我们可以为使用计算力的每一位依赖人搭配一位与其需求相匹配的工作员。