Tensor Gem (Native Matrix/Tensor Extension)

This repository provides a native Ruby extension for fast matrix/tensor operations, packaged as the tensor gem. It uses OpenMP for parallelism and can be linked against OpenBLAS for better numeric performance.

Installation

• Build the extension and gem:
  • ruby extconf.rb
  • make
  • gem build tensor.gemspec
• Install locally:
  • gem install tensor-0.1.0.gem

In Ruby:

require "tensor"

This defines a top-level Tensor class backed by the C extension (an order-2 tensor). Matrix is provided as a backwards-compatible alias.

Basic Usage

m1 = Tensor.new(2, 2, dtype: "float32")
m2 = Tensor.new(2, 2, dtype: "float32")

product = m1.multiply(m2)     # or m1.matmul(m2)
relu    = product.relu
softmax = product.softmax
loss, grad = softmax.cross_entropy_loss([0, 1])

• Supported dtypes for most math ops: "float32", "float64".
• Integer dtypes ("int16", "int8") are supported for storage, indexing, and conversion, but many numeric operations will raise ArgumentError if used with integer matrices.

N-D Tensor API

• Create tensors by shape:
  • Tensor.zeros(shape: [batch, channels, height, width], dtype: "float32")
  • t.shape #=> [b, c, h, w], t.rank, t.size.
• Build from nested Ruby arrays:
  • Tensor.from_array(nested, "float32") where nested is a rectangular N-D array.
• N-D indexing:
  • t[batch, channel, i, j] to get/set a scalar element (negative indices supported).
  • t.to_a returns nested arrays matching t.shape.

For now, higher-order tensors primarily support storage, indexing, and conversion; linear algebra and neural-network operations (e.g., matmul, softmax, cross_entropy_loss) are defined for 2D tensors, and batched N-D versions can be added next.

Single- vs Multi-threaded Execution (OpenMP)

• The C code uses OpenMP (#pragma omp parallel for) for:
  • Matrix multiplication (multiply / matmul)
  • Element-wise ops (subtract, hadamard, scale, ReLU, ReLU grad)
  • Cross-entropy loss and gradient
• The number of threads is chosen as:
  • OMP_NUM_THREADS (if set by your environment/OpenMP runtime), otherwise
  • omp_get_max_threads() at runtime, or a fallback of 16 when OpenMP is not available.

To force single-threaded execution (for debugging or deterministic benchmarking), set:

export OMP_NUM_THREADS=1

To allow more parallelism (on a machine with many cores), set for example:

export OMP_NUM_THREADS=8

Note: the extension currently runs heavy OpenMP regions under Ruby’s Global VM Lock (GVL), so long-running operations can block other Ruby threads even though native work is parallelised internally. Use separate Ruby processes (e.g., via Process.fork or job systems) for true multi-process parallelism.

Build Requirements

• Ruby with development headers (e.g., via rbenv/rvm and ruby-build).
• A C toolchain:
  • macOS: Xcode Command Line Tools (xcode-select --install).
  • Linux: gcc, make, and standard build tools.
• Optional but recommended:
  • OpenMP runtime (libomp on macOS, libgomp/libomp on Linux) for multi-threaded kernels.
  • OpenBLAS for potential BLAS-level optimizations.

On macOS with Homebrew:

brew install libomp openblas

The extconf.rb script will auto-detect these libraries via pkg-config and common Homebrew paths. If they are missing, the gem will still attempt to build, but some optimizations (OpenMP/BLAS) may be disabled.