[RFC]: achieve ndarray API parity with built-in JavaScript arrays

[headlessNode]

Full name

Muhammad Haris

University status

No

University name

Air University, Islamabad, Pakistan

University program

Bachelor of Mechanical Engineering

Expected graduation

Graduated

Short biography

I have always been interested in computers and engineering. My interest in programming began during my bachelors degree when I got introduced to computational fluid dynamics (CFD) software and how they use numerical methods to approximate real-life engineering problems. My final-year research was on a two-phase fluid flow analysis in a Gas Dynamic Virtual Nozzle (GDVN). The analysis was done using OpenFOAM, an open-source computational fluid dynamics (CFD) software written in C++. Apart from creating the nozzle geometry and handling post-processing, I had to fine-tune the numerical solvers using C++ to fit the needs of our research.

After graduation, I started a self-learning journey of software development. I learned C++ and assembly language (ASM) for embedded software and reverse engineering. Later, I shifted my focus to web development, becoming proficient in HTML, CSS, JavaScript, Node.js, React, and SQL. In 2024, I discovered the Google Summer of Code program, which introduced me to stdlib. The work that stdlib is doing to bring numerical and scientific computing to the web, reignited my passion in scientific computing.

Currently, I am focused on increasing my knowledge and expertise in Software/Web development while pursuing my passion for engineering and scientific computation through my work with stdlib.

Timezone

PKT (UTC+5)

Contact details

[email protected], [email protected]

Platform

Linux

Editor

I use Visual Studio Code because I find it intuitive. One of my favorite features is the built-in Emmet abbreviation support, which makes working with HTML and JSX much faster. I also like the built-in debugger terminal, it makes it easy to inspect variables, test functions, and diagnose errors.

Programming experience

As part of my self-learning software engineering journey I have developed several web-based projects of varying complexities. Below are the projects which represent a key milestone in my learning process:

Tic-Tac-Toe: A two-player interactive game built using HTML, CSS and JavaScript while adhering to SOLID design principles.
GitHub: Link

Weather app: A weather app which fetches and displays real-time weather data. This project focused on API integration and asynchronous operations.
GitHub: Link

BattleShip: A single player battleship game featuring an AI opponent, developed using Test-Driven Development (TDD) to increase code reliability and maintainability.
GitHub: Link

Eunoia: A mock online clothing store. built using modern frameworks and libraries including Vite/React, React Router, Tailwind CSS, GreenSock Animation Platform(GSAP). This was a capstone project for my frontend development journey.
GitHub: Link

WhisperSpace: A full-stack web application that allows users to share thoughts, anonymously. Features a secure user authentication using Passportjs and bcryptjs, with a membership system that allows only the members to view authors of the posts.
GitHub: Link

JavaScript experience

I have been using JavaScript as my main programming language for quite some time now. I’ve used it in my personal projects as well as in my open-source contributions. What I really like about JavaScript is its asynchronous nature, which allows it to handle multiple tasks without pausing execution making it ideal for handling user interactions, API calls and real-time events.

However, when handling data from external APIs, JavaScript callbacks allow external entities control over program’s execution flow, creating a security risk. While the introduction of the Promise and async/await syntax has improved the readability and error handling, I think there is still room for improvement in terms of security and control when working with external APIs.

Node.js experience

I have primarily used Node.js to set up servers and REST APIs. I've built a couple full-stack projects using Node.js with Express.js, where I designed and implemented both the backend and frontend. The projects involved handling user authentication, managing a database, and exposing various endpoints. Also, as part of my contributions to stdlib, I developed the initial code infrastructure using Node.js with Fastify for the test code coverage website.

Furthermore, I am currently working on a personal portfolio and a blog website. The backend includes a content management system (CMS) which will allow easy management of posts and portfolio content. This is still a work-in-progress.

C/Fortran experience

My experience in C comes from my contributions to stdlib, where I implemented C ndarray APIs for 60+ packages in the extended BLAS namespace. I don’t have any experience with Fortran but I don’t expect it to be difficult to learn.

Interest in stdlib

In 2024, I discovered the Google Summer of Code program through which I was introduced to stdlib. Due to my background in engineering and interest in scientific computing I was immediately interested in stdlib and the work that is being done to bring numerical and scientific computing to the web.

Within stdlib, I find the Basic Linear Algebra Subprograms (BLAS) and N-dimensional arrays (ndarrays) API particularly interesting because of their use in approximating the solutions to real world engineering problems.

Beyond the technical aspects, I am grateful to be a part of a supportive community that has helped me in my journey to transition to software development.

Version control

Yes

Contributions to stdlib

I began contributing to stdlib in March 2024. Since then, I have authored 211 merged pull requests and 11 pull requests are currently open. These contributions are spread across various namespaces and repositories at stdlib:

• Contributions to the main stdlib repository
• Contributions to the stdlib website
• Contributions to the code coverage website

stdlib showcase

Repository: Link

Summary: For the stdlib showcase project, I decided to use the @stdlib/plot API to visualize the sine wave generated by the @stdlib/simulate-iter-sinewave, based on the user-provided parameters.

The settings modal allows users to input the wave amplitude, period, iterations and sampling rate. The amplitude, period and iterations are used by the @stdlib/simulate-iter-sinewave to generate the sine wave. Additionally, I implemented a custom class to manage the updates to plot parameters and plot size. The plot size and the number of ticks on the x and y axis are adjusted based on screen size.

Goals

My goal is to achieve full API parity between stdlib ndarrays API and built-in JavaScript arrays. This includes implementing all of the 32 remaining functions listed in this tracking issue.

Each implementation will be a complete package, including documentation, benchmarks, tests, TypeScript declarations for type safety and the core JavaScript implementation. The functions will have a base implementation under ndarray/base/* and a top-level user-facing implementation under ndarray/*. The top-level functions will provide a simplified API by abstracting the underlying base logic.

Whether a base implementation is needed depends on the complexity of the function, the need for specialized kernels, and the presence of usable base-level utilities. For example, a base implementation for the function toReversed already exists at ndarray/base/toReversed, so only a top-level ndarray/toReversed implementation will be developed.

Additionally, for performance intensive operations such as sorting, searching and reductions, C implementations will be developed and corresponding Node.js native add-ons at the base-level for accelerated performance. As of writing this application, the native addons will be added for the following functions:

• ndarray/base/fill
• ndarray/base/find
• ndarray/base/excludes
• ndarray/base/reduce
• ndarray/base/some
• ndarray/base/someBy
• ndarray/base/none
• ndarray/base/any
• ndarray/base/none
• ndarray/base/sort

If time allows, additional functions may be considered for native support based on further research.

Why this project?

JavaScript lacks a general purpose and cache efficient multi-dimensional array API. While stdlib provides an ndarray API to address this limitation, further work is needed to drive its adoption in the JavaScript ecosystem. Achieving full API parity with built-in JavaScript arrays, will improve the usability and adoption of stdlib ndarrays, making numerical computing more accessible in JavaScript. This aligns with my interests in computing and engineering, particularly in developing and improving the tools for scientific computing. Also, the functions developed in this project will benefit a wide range of developers and leave a lasting impact in the JavaScript ecosystem—which is what really motivates me.

Qualifications

I have been an active contributor at stdlib for a year, gaining an understanding of its conventions and workflow. I have experience in working with the stdlib ndarray API through my contributions which includes, adding tests for low-level kernels of the unary and nullary functions, adding base implementations of map, toReversed and fill, adding top-level implementations of map and forEach, and adding low-level kernels for 3d to 10d ndarrays for unary-reduce-subarray. This experience allowed me to gain a deeper understanding of the stdlib ndarray API due to which I am confident that I can successfully achieve the aims of this project.

Prior art

Python is the widely used language for multi-dimensional scientific computing operations, thanks to libraries like NumPy, which provides the np.ndarray API with an extensive list of functions for operating on ndarrays.

In JavaScript, apart from the functions offered by the stdlib ndarray API, libraries like TensorFlow.js and scijs also provide multi-dimensional array-like data structures. TensorFlow.js offers an immutable multi-dimensional array tf.tensor API primarily for machine learning applications. It includes functions for element-wise operations, linear algebra, slicing, reshaping and reduction operations. Whereas, scijs offers a mutable ndarrays API for scientific computation with functions for filling, row-wise concatenation, column-wise concatenation and element-wise operations on ndarrays as well as linear algebra and signal processing.

While these libraries provide useful functions that may serve as a reference for our project, JavaScript still lacks a standardized ndarray API that allows operations similar to built-in arrays.

Commitment

I don’t have any major commitments for the summer of 2025. As such, I can dedicate 30 - 40 hours per week on this project.

Schedule

Currently, there are 32 remaining functions to achieve the API parity as outlined in the tracking issue. The complexity of their implementation varies—some will be simple wrappers around existing functions. For example, ndarray/base/slice-fill will be a wrapper around ndarray/base/slice and ndarray/base/fill. Others require utility functions to be implemented beforehand. For example, before working on ndarray/base/every-by, we first need to develop a related reduction function ndarray/base/unary-reduce-subarray-by.

Additionally, some functions also require writing kernels for specific dimensions up to 10d with an additional nd kernel for higher dimensions. This significantly increases the time required for developing the full package, which apart from the core-logic will include, tests, benchmarks, and documentation.

Based on these factors, the following is my proposed schedule for the project, assuming a 12-week schedule:

• Community Bonding Period: During this period, my focus will be to develop the utility functions which would serve as the foundation for the functions required to achieve API parity. As of writing this application, the following utility functions need to be implemented, but the need for other may arise as the work progresses:
  • ndarray/base/unary-reduce-subarray-by: This will serve functions like ndarray/someBy, ndarray/anyBy,
  • ndarray/everyBy, ndarray/noneBy.
  • ndarray/base/assign-scalar: This will serve functions like ndarray/full, ndarray/ones, ndarray/nans.
  • Search: We will need to implement utility functions based on a specific search algorithm however this needs
    further research.
  • Sort: We will need to implement utility functions based on a specific sort algorithm, this also requires further
    research.

Apart from the utility functions, I also aim to have implemented the base-level ndarray/base/* implementations for 5-6 functions mentioned in the tracking issue. This will give us a nice head start before the coming weeks.

• Week 1 to Week 6: During these weeks I aim to implement and open PRs for the base ndarray/base/* implementations for all the remaining functions in the tracking issue. The final PR for each function implementation will be a whole package containing benchmarks, documentation, tests and the main JavaScript implementation and in some cases C implementation for native add-ons support. I expect that these PRs will also be merged during this time, after implementing the requested changes that may be highlighted by code review.

• Week 6 to Week 12: During these weeks I aim to implement and open PRs for the top-level ndarray/* implementations for the related base implementations implemented in previous weeks. These would also be complete packages containing benchmarks, documentation, tests and the main implementation. I expect that these PRs will also be merged during this time, after implementing the requested changes that may be highlighted by code review.

• Final Week: In the final week I will focus on completing any remaining work that needs to be done for the successful completion of this project this may include project report and getting any open PRs merged.

Post-GSoC: If some functions for achieving API parity are left, I will continue to work on them. I also aim to work on adding functions related to approximating the solutions for differential equations and be an integral part of the stdlib community.

Notes:

• The community bonding period is a 3 week period built into GSoC to help you get to know the project community and participate in project discussion. This is an opportunity for you to setup your local development environment, learn how the project's source control works, refine your project plan, read any necessary documentation, and otherwise prepare to execute on your project project proposal.
• Usually, even week 1 deliverables include some code.
• By week 6, you need enough done at this point for your mentor to evaluate your progress and pass you. Usually, you want to be a bit more than halfway done.
• By week 11, you may want to "code freeze" and focus on completing any tests and/or documentation.
• During the final week, you'll be submitting your project.

Related issues

Tracking Issue: stdlib-js/stdlib#2656

Project Idea: #33

Checklist

• I have read and understood the Code of Conduct.
• I have read and understood the application materials found in this repository.
• I understand that plagiarism will not be tolerated, and I have authored this application in my own words.
• I have read and understood the patch requirement which is necessary for my application to be considered for acceptance.
• I have read and understood the stdlib showcase requirement which is necessary for my application to be considered for acceptance.
• The issue name begins with [RFC]: and succinctly describes your proposal.
• I understand that, in order to apply to be a GSoC contributor, I must submit my final application to https://summerofcode.withgoogle.com/ before the submission deadline.

[headlessNode]

cc @kgryte @Planeshifter

[kgryte]

@headlessNode Thank you for opening this RFC. A few comments:

1. ndarray/base/find, ndarray/base/reduce, and ndarray/base/some-by will not need native add-ons. The reason being is that any API which leverages JavaScript callbacks where we need to pass in indices, the input ndarray, and support an execution context can only happen in JS-land. For ndarray APIs in which callbacks can have equivalent JS-C implementations (e.g., as intended in ndarray/base/unary), we can have both C and JS implementations. Hence, while we won't have C implementations and thus native add-on support for ndarray/base/every-by, we will be able to have a C implementation and will be able to support native add-ons for ndarray/base/unary-every. I know this is slightly confusing, but it mainly boils down to: can we support the same semantics in both C and JS? If yes, then add a C implementation. If not, then only JS.
2. For package names, we don't use camelcase, but snakecase. Hence, @stdlib/ndarray/base/someBy should be @stdlib/ndarray/base/some-by.