Solpipe.io

Shared-buffer trading Host Story

Thu, 10 Oct 2024 00:00:00 +0000

Shared-buffer Host #

The “Shared-buffer” host, a specialized geyser plugin that allows trading bots to run directly in the Validator, is a tool we have developed and are offering to Validators interested in maximizing their validator earnings. By running in the validator, traders are getting unparalleled low-latency

On solpipe #

With Solpipe ‘s decentralized automated auction system, validators run our plugin and traders can pay for a spot in the host to run their bot guests. Solpipe makes it possible to rent Validator access and run your bot directly in Validators anywhere, anytime. Traders can pick and choose where to connect.

Tx-Proc Story

Tue, 19 Mar 2024 00:00:00 +0000

The Solana Transaction Processor, or Tx-Proc, is a server designed to write to the Solana Blockchain network.

This “story” demonstrates how by running the Solpipe server alongside your Validator, you can effectively monetize your Solana blockchain operations.

Solana Tx-Proc is a open-source projects. Utilize this library to experience firsthand how simple it is to monetize a server on Solpipe. Learn the necessary steps by visiting the following link: Run a Pipeline

Download API Specifications

Mon, 01 Jan 0001 00:00:00 +0000

All connections in Solpipe are established through an interface created by a “Protocol Buffer” (protobuf).

All users must download the marketplace protobuf to initiate and receive calls from other users.

Download #

To download the marketplace and the necessary protobuf file(s) from the Solpipe Marketplace Explorer or retrieve the “marketplace id” and execute the following:

solpipe market download  <marketplace-id>
ls

Untar Market files #

tar -xvzf market.tar.gz -C .

Next Steps #

As a pipeline or bidder user, your server code must be compiled to align with the marketplace protobuf specifications.

Marketplace FAQ

Mon, 01 Jan 0001 00:00:00 +0000

How do I retrieve my marketplace ID? #

Use the market hash command with a boolean flag “id”.

Execute the following command in the directory that holds your marketplace files:

solpipe market hash --id

How to I retrieve marketplace account information? #

To print the relevant bidder agent details execute the following:

solpipe view market <marketplace-id> [flags]

Help: Arguments and Flags

marketplace-id: Specify your marketplace ID.

[flags]
show-full: Include the show-full boolean flags to print the entire marketplace state.

Quick Start

Mon, 01 Jan 0001 00:00:00 +0000

For a comprehensive understanding, see how to Run a Bidder for detailed information.

Bidder Quick Start #

See how to install Solpipe executables here(if applicable).

Compile Server

Assure you have downloaded the necessary protocol buffer(s) (.proto) files from the marketplace and compiled into your server code. See how to Implement API Specs

SafeJar

To participate in the continuous bandwidth auction on Solpipe, your company must initialize a ‘Jar’ and ‘Delegation account’ using the SafeJar Program.

Create a directory #

mkdir my-bidder
cd my-bidder

Start your Proxy Daemon #

 solpipe bidder proxy  --listen=tcp://localhost:34043

Add a Marketplace and Run Bidder #

solpipe bidder market <marketplace-id> <jar-id> ./rule.set

Quick Start

Mon, 01 Jan 0001 00:00:00 +0000

For a comprehensive understanding, please refer to Running a Pipeline for detailed information.

Pipeline Quick Start #

See how to install Solpipe executables here(if applicable).

Compile Server

Assure you have downloaded the necessary protocol buffer(s) (.proto) files from the marketplace, and compiled into your server code. See how to Implement API Specs

SafeJar

To participate in the continuous bandwidth auction on Solpipe, your company must initialize a ‘Jar’ and ‘Delegation account’ using the SafeJar Program.

Initialize Pipeline Directory #

mkdir my-pipeline
cd my-pipeline
solpipe pipeline init <marketplace-id> . --window=1m --create-jar

Note

Personalize your pipeline, by making changes to the bot.json file. Refer to Pipeline Bot for further details.

Create a Pipeline Instance #

solpipe pipeline create ./bot.json

Turn on your Bot #

solpipe pipeline bot ./bot.json

Turn on your Relay #

Note

Personalize your pipeline, by making changes to the relay.json file. Refer to Pipeline Relay for further details.

Launch a new terminal to run your relay.

solpipe pipeline relay ./relay.json

Run A Pipeline

Tue, 20 Aug 2024 00:00:00 +0000

The Solana Transaction Processor, or Tx-Proc, is a server designed to write to the Solana Blockchain network.

Solana Tx-Proc is a open-source projects. Utilize this library to experience firsthand how simple it is to monetize your Solana Validator.

Throttling with Solpipe #

Solpipe implements bandwidth throttling for Solana validators, streamlining the process of managing bandwidth allocation. Instead of manually configuring bandwidth prices, Solpipe automates pricing based on the validator’s bandwidth constraints/stake, saving validators time and effort. Pricing is dynamically determined by demand and the available network resources. People can access lead validator bandwidth not by stake or fees (tips) but by a price set by demand, ensuring fair and efficient access to network resources based on market value.

Pipeline FAQ

Thu, 05 Jan 2017 00:00:00 +0000

How to retrieve a marketplace ID? #

You can locate the marketplace ID for the market where you intend to run a pipeline on the Solpipe.io marketplace explorer page.

How do i download Protobuf(s) from a marketplace? #

solpipe download market <marketplace-id>
ls

Help: Arguments and Flags

marketplace-id: Specify your marketplace ID.

This will give you all relevant marketplace files needed to compile your server.

Quick Start

Thu, 05 Jan 2017 00:00:00 +0000

For a comprehensive understanding, please refer to Creating a Marketplacefor detailed information.

Marketplace Quick Start #

See how to install executables here (if applicable).

Initialize your directory #

mkdir my-market
cd my-market
solpipe market init .

Note

To personalize your marketplace, you’ll need to make changes to the marketplace Files. Refer to Tailor your Specialty Marketplace.

Attach funding source #

Choose an SPL Token account for your mint and attach a A hardware wallet designed for storing and managing cryptocurrencies securely. ledger nano to fund the transactions.

Note

Assign a unique key ID for your admin-key and fee-payer.

For USDC specify the following key:

export MINT=EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v
SOLPIPE_ADMIN_KEY=usb://ledger?key=3000

export MINT=So11111111111111111111111111111111111111112
SOLPIPE_ADMIN_KEY=usb://ledger?key=3000

Compile Protobuf Libraries

Mon, 01 Jan 0001 00:00:00 +0000

Compile your code #

Retrieve the necessary protocol buffer(s) (.proto) files, and import the libraries into your server code. This step involves incorporating the provided files into your existing codebase, enabling your server to interact with your chosen marketplace.

Protocol Buffers (Protobuf) #

Protocol Buffers or Protobufs, are a versatile, open-source tool for structuring data in a way that facilitates easy serialization. This cross-platform solution is particularly handy for developing applications that communicate seamlessly over networks or for efficiently storing data. With Protobufs, data can be encoded in a compact and efficient binary format, enhancing both speed and resource utilization in data exchange processes.

Install with Linux

Mon, 01 Jan 0001 00:00:00 +0000

Before you start exploring Solpipe, install the ‘solpipe-terminal’ directory to execute both ‘solpipe’ and ‘safejar’ commands.

Debian (Bookworm or Newer) #

To install using Debian Bookworm (direct) execute the following commands:

sudo apt-get update
sudo apt-get install -y wget gnupg 
wget -O- https://noncepad.com/keys/stable.bookworm.sources > /tmp/x 
sudo mv /tmp/x  /etc/apt/sources.list.d/noncepad.stable.sources
sudo apt-get update 
which solpipe

Docker #

To install in a docker container, execute the following command:

docker build -t solpipe:1 https://noncepad.com/dev/Dockerfile

Next run the container with the following commands:

Running a Bidder

Mon, 01 Jan 0001 00:00:00 +0000

Here outlines all the procedures needed to deploy a Bidder. Start running your bidder with the following commands: Quick Start.

Compiling to Your Server #

To initiate a bidder in a Solpipe marketplace, integrate the marketplace libraries into your code. See how to Implement API Specs.

SafeJar #

To participate in the continuous bandwidth auction on Solpipe, your company must initialize a ‘Jar’ and ‘Delegation account’ using the SafeJar Program. Noncepad has established SafeJar, a treasury service that facilitates the allocation of working capital, governed by a rule-set, from the CFO/financial department to an employee.

To start using SafeJar, please visit the website at safejar.io.

Running a Pipeline

Mon, 01 Jan 0001 00:00:00 +0000

Here outlines all the procedures needed to deploy a pipeline. Start running your pipeline with the following commands: Quick Start.

Creating a Pipeline and How it Works #

To create a pipeline you will need to configure two integral components: a “bot” and a “relay”. The bot assumes the role of creating and overseeing the pipeline, while the relay is responsible for the communication between the bidders and pipelines.

Once your pipeline is activated, bidders can submit bids for a share of the server bandwidth you operate, securing a proportion based on the strength of their bid.

Informing Bidders

Wed, 21 Aug 2024 00:00:00 +0000

Marketing Your Pipeline #

To successfully market your pipeline and attract bidders, it’s essential to provide them with clear instructions and tools for accessing your TxProc server bandwidth. This section outlines the key information and resources you should provide to potential bidders.

Key Information:

Pipeline Capabilities: Clearly describe your pipeline’s capabilities, including its bandwidth capacity, latency, and any specific features or optimizations.
Pricing Model: Explain Solpipe Pricing Model.
Solpipe Bidder CLI and tooling explanation

Additional Resources:

Init Pipeline with SafeJar

Mon, 01 Jan 0001 00:00:00 +0000

For more information on the SafeJar program visit Safejar.io.

SafeJar #

Noncepad has established SafeJar, a treasury service that facilitates the allocation of working capital, governed by a rule-set, from the CFO/financial department to an employee. To participate in the continuous bandwidth auction on Solpipe, your company must initialize a ‘Jar’ and ‘Delegation account’ using the SafeJar Program. This can be done one of three ways: 1. Create a default jar/delegation account. 2. Initialize a pipeline with a ‘Jar Account’ and default delegation account. 3. Initialize a pipeline with a ‘Delegation Account’.

Pipeline Capacity

Mon, 01 Jan 0001 00:00:00 +0000

Relay retrieves capacity from backend server as a gRPC stream.

Incorportate A Capacity Endpoint #

Implement a gRPC endpoint defined by the following protobuf:

service Capacity{
    rpc OnStatus(Empty) returns (stream CapacityStatus) {}
}

message Empty{}

message CapacityStatus{
    float utilization_ratio = 1;
}

There are 3 methods:

A worker-pool
A CPU meter
A bespoke metering tool

Worker Pool Method #

Define your backend capacity with Noncepad’s open source worker-pool.

CPU Meter #

Monitor your available capacity with Noncepad’s open source CPU meter.

Bidder FAQ

Thu, 05 Jan 2017 00:00:00 +0000

How to I retrieve bidder account information? #

To print the relevant bidder agent details execute the following:

solpipe view agent <agent-id> [flags]

Help: Arguments and Flags

agent-id: Specify your bidder ID.

[flags]
show-bids: Include the show-bids boolean flags to print the entire bidder agent state including all bids.

Creating a Marketplace

Mon, 01 Jan 0001 00:00:00 +0000

Here outlines all the procedures needed to deploy a marketplace. Initiate your marketplace with the following commands: Quick Start.

Initialize a Directory #

Executing this command will initialize the libraries required for the upload process in a new directory:

solpipe market init <directory>

solpipe market init .

Help: Arguments and Flags

directory: Specify and name a directory to save your ‘marketplace’ library. Use for example: “./my-market”, or to specifiy the currect directory use: “.”

Understanding bot.json

Mon, 01 Jan 0001 00:00:00 +0000

Guidelines for the configuration of the pipeline bot.

The sample template bot.json file will look similar to the following and should be configured for your specialty pipeline:

{
"Controller": "8XyZu8x7Wy4Ka7o7rFPbQcaDx4z8xJsoejnuFk6qAbzU",
"Authorizer": "E5jgnJbcE4pnJfFAXkrPiHpEmh2m2QhFFXoGnuhH98At",
"Vault": "7DieusiSC4tqW6v6HsRAQiSEww7Ecn42v7Xdzqx99My9",
"TickSize": 1,
"Scoop Delay" 2,
"RefundSpace": 10,
"BidSpace": 10,
"PeriodLength": 600,
"MaxAuctionLength": 600,
"MaxPayoutCount": 10
}

Controller #

The controller is marketplace id and the unique identifier from the selected marketplace of your pipeline.

Authorizer #

The authorizer is an essential public-private key-pair for generating signatures while running the bot. The authorizers pubkey, specified in the bot, is used by bidders when connecting to the pipeline.

Understanding relay.json

Thu, 05 Jan 2017 00:00:00 +0000

Guidelines for the configuration of the pipeline relay file.

The sample template relay.json file will look similar to the following and should be configured for your special pipeline:

{
	"non-bidder:false
	"usagelua": /tmp/usage.lua
	"window":300000000000
	"pipeline": "5HV9d1WeMDVcYLD1tWgtgApR4hB998LoXK4xB3xaRRKL",
	"backends":[
		{
		"services":["testecho.Echo"],
		"address":"tcp:////localhost:30000"
		}
	],
	"endpoints":[
		{
		"routable":["tcp://2405:1200:e08d:c900::dc04:9ac4:50051"],
		"listen":"tcp://:50051"
		}
	]

}

non-bidder #

The ’non-bidder’ parameter in our continuous bandwidth auctioning software is a boolean setting that determines the accessibility of ‘idle pipeline capacity’ for potential bidders. When set to ’true’, idle capacity is openly available for bidders to connect to without the need for payment. Conversely, setting it to ‘false’ (default) restricts access to idle capacity, requiring bidder accounts to pay for service.

Getting Started with Smart Contracts on Solana

Mon, 01 Jan 0001 00:00:00 +0000

Take some time to learn about Solana smart contracts before using our system, as financial transactions occur on the blockchain.

Benefits of Decentralized Transactions #

Solpipe’s integration with Solana’s decentralized ecosystem empowers providers to control their bandwidth resources without relying on centralized intermediaries. The continuous auction ensures fair pricing that accurately reflects market forces, ensuring both pipelines and bidders receive fair value. The platform’s integration with Solana makes it easy for developers to access and utilize Solpipe’s functionality, unlocking the full potential of decentralized bandwidth monetization.

GPU Bandwidth Marketplace

Wed, 05 Feb 2025 00:00:00 +0000

Solpipe is creating a decentralized GPU marketplace where GPU owners can rent out their idle resources and AI developers can tap into affordable, scalable compute power.

The Market #

The demand for high-performance GPUs is skyrocketing, fueled by the rapid advancements in artificial intelligence, machine learning, and high-performance computing. According to industry reports, the global AI hardware market is projected to exceed $300 billion by 2030, with GPUs playing a critical role in powering deep learning models, real-time data processing, and complex simulations.

Geyser Story

Sun, 26 Mar 2023 00:00:00 +0000

The Solana Geyser is a server designed to read the Solana Blockchain network.

This “story” demonstrates how by running the Solpipe server alongside your Validator, you can effectively monetize your Solana blockchain operations.

Solana Geyser is a open-source projects. Utilize this library to experience firsthand how simple it is to monetize a server on Solpipe. Learn the necessary steps by visiting the following link: Run a Pipeline

Geyser #

Auctioned Units: Solana Blockchain Live Chain state

Run A Pipeline

Mon, 20 Mar 2023 00:00:00 +0000

FFmpeg Story

Tue, 28 Feb 2023 00:00:00 +0000

FFmpeg-Converter is a server designed to transform Blender files (.blend) into diverse video formats.

“The FFmpeg Story” documents the inception of the idea and outlines how I successfully monetized my tool using Solpipe.

The FFmpeg-Converter is the open-source project I wrote. Utilize this library to experience firsthand how simple it is to monetize a server on Solpipe. Learn the necessary steps by visiting the following link: Test Solpipe

Harvesting the Gold from Within #

Introduction #

In today’s rapidly evolving business landscape, organizations strive to optimize internal processes for greater efficiency and productivity. Like many, I was given a task from my boss, saw the benefits in automating said task and developed a tool to do so. Seeing its potential, I was able to utilize Solpipe to easily transform my tool into valuable assets for the broader market.

FFmpeg in Go

Mon, 27 Feb 2023 00:00:00 +0000

FFmpeg-Converter is a server designed to transform Blender files (.blend) into diverse video formats.

“FFmpeg in Go” outlines the initial step I took before developing my tool.

The FFmpeg-Converter is the open-source project I wrote. Utilize this library to experience first-hand how simple it is to monetize a server on Solpipe. Learn the necessary steps by visiting the following link: Test Solpipe

When starting the FFmpeg converter API development, my first challenge was verifying if I could execute an FFmpeg command from a Go package. By wrapping an FFmpeg conversion command within a Go test, I effectively validated the functionality.

FFmpeg over gRPC

Sun, 26 Feb 2023 00:00:00 +0000

“FFmpeg with gRPC” details how I employed a “Protocol Buffer” to serve my tool over the internet.

To handle internet requests, I set up a gRPC connection, “A high performance, open source universal RPC framework”. To serve requests with gRPC define the service interface and message types using a Protocol Buffer (protobuf). The protobuf and go code must work together.

Throttling with Solpipe

Sat, 25 Feb 2023 00:00:00 +0000

“FFmpeg on Solpipe” details my rationale for selecting Solpipe as my monetization tool and outlines the necessary steps to implement it.

Throttling API based bandwidth is akin to directing cars into two lanes to regulate traffic flow effectively.

Price by throttling not per request #

My final step to monetization was solved with Solpipe. The traditional route for monetizing API based bandwidth can be very complex. With Stripe or AWS it involves setting prices per request which can lead to unpredictable costs and inefficient resource allocation. However, Solpipe’s more innovative approach of throttling offers a more transparent and cost-effective solution. Instead, pricing adjusts dynamically in response to demand and available resources.

Run a Pipeline

Mon, 20 Feb 2023 00:00:00 +0000

FFmpeg-Converter is a multimedia server designed to transform Blender files (.blend) into diverse video formats.

Explore the process of monetizing your server with Solpipe by deploying a pipeline using the FFmpeg libraries!

Setup your Environment #

Before getting started make sure you have installed both blender and ffmpeg executables.

Download Solpipe CLI: Solpipe Install.

Run a Server #

This project is open source. Download the FFmpeg-Converter source code and run the server.

Search Results

Mon, 01 Jan 0001 00:00:00 +0000