API Reference

AnalyticOpt{
    T<:Real,
    V<:AbstractArray{T},
    U<:AbstractArray{T},
    A<:AbstractArray{T},
    S<:AbstractVector{<:Hook},
} <: SemioticOpt.OptAlgorithm

Optimise the indexing reward analytically.

Fields

• x::V is the current best guess for the solution. Typically zeros.
• Ω::U is the allocation vector of other indexers.
• ψ::A is the signal vector.
• σ::T is the stake.
• hooks::S are the hooks

Example

julia> using AllocationOpt
julia> using SemioticOpt
julia> x = zeros(2)
julia> Ω = [1.0, 1.0]
julia> ψ = [10.0, 10.0]
julia> σ = 5.0
julia> alg = AllocationOpt.AnalyticOpt(;
           x=x, Ω=Ω, ψ=ψ, σ=σ, hooks=[StopWhen((a; kws...) -> kws[:i] > 1)]
       )
julia> f = x -> x  # This doesn't matter. `f` isn't used by the algorithm.
julia> alg = minimize!(f, alg)
julia> SemioticOpt.x(alg)
2-element Vector{Float64}:
 2.5
 2.5

 allocatablesubgraphs(s::FlexTable, config::AbstractDict)

For the subgraphs s return a view of the subgraphs on which we can allocate.

julia> using AllocationOpt
julia> using TheGraphData
julia> s = flextable([
            Dict("ipfsHash" => "Qma", "signalledTokens" => 10,),
            Dict("ipfsHash" => "Qmb", "signalledTokens" => 20),
            Dict("ipfsHash" => "Qmc", "signalledTokens" => 5),
       ])
julia> config = Dict(
            "whitelist" => String["Qmb", "Qmc"],
            "blacklist" => String[],
            "frozenlist" => String[],
            "pinnedlist" => String[],
            "min_signal" => 0.0
)
julia> fs = AllocationOpt.allocatablesubgraphs(s, config)
FlexTable with 2 columns and 2 rows:
     signalledTokens  ipfsHash
   ┌──────────────────────────
 1 │ 20               Qmb
 2 │ 5                Qmc

allocate_action(::Val{:actionqueue}, a::FlexTable, t::FlexTable, config::AbstractDict)

Create and push allocate actions to the action queue.

julia> using AllocationOpt
julia> using TheGraphData
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa")
        ])
julia> t = flextable([
    Dict("amount" => "1", "profit" => "0", "ipfshash" => "Qma"),
    Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"),
])
julia> config = Dict(
    "indexer_url" => "http://localhost:18000"
)
julia> TheGraphData.client!(config["indexer_url"])
julia> AllocationOpt.allocate_action(Val(:actionqueue), a, t, config)
1-element Vector{Dict{String, Any}}:
 Dict("amount" => "2", "priority" => 0, "status" => AllocationOpt.queued, "source" => "AllocationOpt", "reason" => "Expected profit: 0", "type" => AllocationOpt.allocate, "deploymentID" => "Qmb", "protocolNetwork" => "mainnet")

allocate_action(::Val{:none}, a, t, config)

Do nothing.

julia> using AllocationOpt
julia> using TheGraphData
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa")
        ])
julia> t = flextable([
            Dict("amount" => "1", "profit" => "0", "ipfshash" => "Qma"),
            Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"),
        ])
julia> AllocationOpt.allocate_action(Val(:none), a, t, Dict())

allocate_action(::Val{:rules}, a::FlexTable, t::FlexTable, config::AbstractDict)

Print the rules that allocates to new subgraphs.

```julia julia> using AllocationOpt julia> using TheGraphData julia> a = flextable([ Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa") ]) julia> t = flextable([ Dict("amount" => "1", "profit" => "0", "ipfshash" => "Qma"), Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"), ]) julia> AllocationOpt.allocate_action(Val(:rules), a, t, Dict()) graph indexer rules set Qmb decisionBasis always allocationAmount 2 1-element Vector{String}: "graph indexer rules set Qmb decisionBasis always allocationAmount 2"

aquery(id::AbstractString)

Return the components of a GraphQL query for active allocations of indexer id.

For use with the TheGraphData.jl package.

julia> using AllocationOpt
julia> id = "0xa"
julia> value, args, fields = AllocationOpt.aquery(id)
("allocations", Dict{String, Union{Dict{String, String}, String}}("where" => Dict("status" => "Active", "indexer" => "0xa")), ["allocatedTokens", "id", "subgraphDeployment{ipfsHash}"])

Extended Help

You can find TheGraphData.jl at https://github.com/semiotic-ai/TheGraphData.jl

available(::Val{:indexer}, x)

The tokens available for the indexer to allocate in table x.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict(
        "stakedTokens" => 10,
        "delegatedTokens" => 20,
        "lockedTokens" => 5,
    ),
])
julia> AllocationOpt.availablestake(Val(:indexer), x)
25.0

bestprofitpernz(ixs::AbstractVector{Integer}, profitmatrix::AbstractMatrix{Real})

Compute the best profit amongst the given ixs given profit matrix p

julia> using AllocationOpt
julia> ixs = Dict([1] => [1], [2] => [2])
julia> profitmatrix = [[2.5 5.0]; [2.5 1.0]]
julia> AllocationOpt.bestprofitpernz.(values(ixs), Ref(profitmatrix))
2-element Vector{NamedTuple{(:profit, :index), Tuple{Float64, Int64}}}:
 (profit = 5.0, index = 1)
 (profit = 6.0, index = 2)

blockissuance(::Val{:network}, x)

The tokens issued per block.

```julia julia> using AllocationOpt julia> using TheGraphData julia> n = flextable([ Dict( "id" => 1, "networkGRTIssuancePerBlock" => 1, "epochLength" => 28, "totalTokensSignalled" => 2, "currentEpoch" => 1, ) ]) julia> AllocationOpt.blockissuance(Val(:network), n) 1

blocksperepoch(::Val{:network}, x)

The number of blocks in each epoch.

```julia julia> using AllocationOpt julia> using TheGraphData julia> n = flextable([ Dict( "id" => 1, "networkGRTIssuancePerBlock" => 1, "epochLength" => 28, "totalTokensSignalled" => 2, "currentEpoch" => 1, ) ]) julia> AllocationOpt.blocksperepoch(Val(:network), n) 28

closeipfs(existingipfs, proposedipfs, frozenlist)

Get the list of the ipfs hashes of allocations to close.

julia> using AllocationOpt
julia> AllocationOpt.closeipfs(["Qma"], ["Qmb"], String[])
1-element Vector{String}:
 "Qma"

configuredefaults!(config::AbstractDict)

Set default values for the config dictionary if the value was not specified in the config file.

Config Specification

• id::String: The ID of the indexer for whom we're optimising. No default value.
• network_subgraph_endpoint::String: The network subgraph endpoint to query. By default, "https://api.thegraph.com/subgraphs/name/graphprotocol/graph-network-mainnet"
• writedir::String: The directory to which to write the results of optimisation. If don't specify readdir, writedir also specifies the path to which to save the input data tables. By default, "."
• readdir::Union{String, Nothing}: The directory from which to read saved data tables. This speeds up the process as we won't have to query the network subgraph for the relevant data. If you don't specify readdir, we will query your specified network_subgraph_endpoint for the data and write it to CSV files in writedir. This way, you can use your previous writedir as your readdir in future runs. By default, nothing
• whitelist::Vector{String}: A list of subgraph IPFS hashes that you want to consider as candidates to which to allocate. If you leave this empty, we'll assume all subgraphs are in the whitelist. By default, String[]
• blacklist::Vector{String}: A list of subgraph IPFS hashes that you do not want to consider allocating to. For example, this list could include broken subgraphs or subgraphs that you don't want to index. By default, String[]
• frozenlist::Vector{String}: If you have open allocations that you don't want to change, add the corresponding subgraph IPFS hashes to this list. By default, String[]
• pinnedlist::Vector{String}: If you have subgraphs that you absolutely want to be allocated to, even if only with a negligible amount of GRT, add it to this list. By default, String[]
• allocation_lifetime::Integer: The number of epochs for which you expect the allocations the optimiser finds to be open. By default, 28
• gas::Real: The estimated gas cost in GRT to open/close allocations. By default, 100
• min_signal::Real: The minimum amount of signal in GRT that must be on a subgraph in order for you to consider allocating to it. By default, 100
• max_allocations::Integer: The maximum number of new allocations you'd like the optimiser to consider opening. By default, 10
• num_reported_options::Integer: The number of proposed allocation strategies to report. For example, if you select 10 we'd report best 10 allocation strategies ranked by profit. By default, 1
• verbose::Bool: If true, the optimiser will print details about what it is doing to stdout. By default, false
• execution_mode::String: How the optimiser should execute the allocation strategies it finds. Options are "none", which won't do anything, "actionqueue", which will push actions to the action queue, and "rules", which will generate indexing rules. By default, "none"
• indexer_url::Union{String, Nothing}: The URL of the indexer management server you want to execute the allocation strategies on. If you specify "actionqueue", you must also specify indexer_url. By default, nothing
• opt_mode::String: We support two optimisation modes. One is "fast". This mode is fast, but may not find the optimal strategy. This mode is also used to the top num_reported_options allocation strategies. The other mode is "optimal". This mode is slower, but it satisfy stronger optimality conditions. It will find strategies at least as good as "fast", but not guaranteed to be better. In general, we recommend exploring config options using "fast" mode first, and then using "optimal" mode to find the optimal allocation. By default, "optimal"
• protocol_network::String: Defines the protocol network that allocation transactions should be sent to. The current protocol network options are "mainnet", "goerli", "arbitrum", and "arbitrum-goerli". By default, "mainnet"
• syncing_networks::Vector{String}: The list of syncing networks to support when selecting the set of possible subgraphs. This list should match the networks available to your graph-node. By default, the list is a singleton of your protocol network

julia> using AllocationOpt
julia> config = Dict{String, Any}("id" => "0xa")
julia> config = AllocationOpt.configuredefaults!(config)
Dict{String, Any} with 16 entries:
  "execution_mode"            => "none"
  "readdir"                   => nothing
  "writedir"                  => "."
  "num_reported_options"      => 1
  "id"                        => "0xa"
  "pinnedlist"                => String[]
  "indexer_url"               => nothing
  "gas"                       => 100
  "allocation_lifetime"       => 28
  "blacklist"                 => String[]
  "verbose"                   => false
  "min_signal"                => 100
  "network_subgraph_endpoint" => "https://api.thegraph.com/subgraphs/name/graphprotocol/graph-network-mainnet"
  "whitelist"                 => String[]
  "protocol_network"          => "mainnet"
  "syncing_networks"          => String["mainnet"]

correcttypes!(i::FlexTable, a::FlexTable, s::FlexTable, n::FlexTable)

Convert all tables to be in GRT.

julia> using AllocationOpt
julia> using TheGraphData
julia> i = flextable([
    Dict(
        "stakedTokens" => "1",
        "delegatedTokens" => "0",
        "id" => "0xa",
        "lockedTokens" => "0",
    ),
])
julia> s = flextable([
    Dict(
        "stakedTokens" => "1",
        "signalledTokens" => "0",
        "ipfsHash" => "Qma",
    ),
])
julia> a = flextable([
    Dict(
        "allocatedTokens" => "1",
        "subgraphDeployment.ipfsHash" => "Qma",
    ),
])
julia> n = flextable([
    Dict(
        "id" => 1,
        "networkGRTIssuancePerBlock" => "1",
        "epochLength" => 28,
        "totalTokensSignalled" => "2",
        "currentEpoch" => 1,
    )
])
julia> i, a, s, n = AllocationOpt.correcttypes!(i, a, s, n)

correcttypes!(::Val{:allocation}, a::FlexTable)

Convert the string currency fields in the allocation table to be in GRT.

julia> using AllocationOpt
julia> using TheGraphData
julia> a = flextable([
    Dict(
        "allocatedTokens" => "1",
        "subgraphDeployment.ipfsHash" => "Qma",
    ),
])
julia> AllocationOpt.correcttypes!(Val(:allocation), a)
FlexTable with 2 columns and 1 row:
     subgraphDeployment.ipfsHash  allocatedTokens
   ┌─────────────────────────────────────────────
 1 │ Qma                          1.0e-18

correcttypes!(::Val{:indexer}, i::FlexTable)

Convert the string currency fields in the indexer table to be in GRT.

julia> using AllocationOpt
julia> using TheGraphData
julia> i = flextable([
    Dict(
        "stakedTokens" => "1",
        "delegatedTokens" => "0",
        "id" => "0xa",
        "lockedTokens" => "0",
    ),
])
julia> AllocationOpt.correcttypes!(Val(:indexer), i)
FlexTable with 4 columns and 1 row:
     stakedTokens  delegatedTokens  id   lockedTokens
   ┌─────────────────────────────────────────────────
 1 │ 1.0e-18       0.0              0xa  0.0

correcttypes!(::Val{:network}, n::FlexTable)

Convert the string currency fields in the network table to be in GRT.

julia> using AllocationOpt
julia> using TheGraphData
julia> n = flextable([
    Dict(
        "id" => 1,
        "networkGRTIssuancePerBlock" => "1",
        "epochLength" => 28,
        "totalTokensSignalled" => "2",
        "currentEpoch" => 1,
    )
])
julia> AllocationOpt.correcttypes!(Val(:network), n)
FlexTable with 6 columns and 1 row:
    totalTokensSignalled  currentEpoch  id  networkGRTIssuancePerBlock  epochLength
┌────────────────────────────────────────────────────────────────────────────────
1 │ 2.0e-18               1             1   1.0e-18                     28

correcttypes!(::Val{:subgraph}, s::FlexTable)

Convert the string currency fields in the subgraph table to be in GRT.

julia> using AllocationOpt
julia> using TheGraphData
julia> s = flextable([
    Dict(
        "stakedTokens" => "1",
        "signalledTokens" => "0",
        "ipfsHash" => "Qma",
        "deniedAt" => 0,
    ),
])
julia> AllocationOpt.correcttypes!(Val(:subgraph), s)
FlexTable with 4 columns and 1 row:
     deniedAt  stakedTokens  signalledTokens  ipfsHash
   ┌──────────────────────────────────────────────────
 1 │ 0         1.0e-18       0.0              Qma

currentepoch(::Val{:network}, x)

The current epoch.

```julia julia> using AllocationOpt julia> using TheGraphData julia> n = flextable([ Dict( "id" => 1, "networkGRTIssuancePerBlock" => 1, "epochLength" => 28, "totalTokensSignalled" => 2, "currentEpoch" => 1, ) ]) julia> AllocationOpt.currentepoch(Val(:network), n) 1

delegation(::Val{:indexer}, x)

The tokens delegated to the indexer in table x.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict(
        "delegatedTokens" => 10,
    ),
])
julia> AllocationOpt.delegation(Val(:indexer), x)
10

deniedat(::Val{:subgraph}, x)

If this value is non-zero, the subgraph doesn't receive indexing rewards.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict("deniedAt" => 10,),
    Dict("deniedAt" => 0,),
])
julia> AllocationOpt.deniedat(Val(:subgraph), x)
2-element view(transpose(lazystack(::Vector{Vector{Int64}})), :, 1) with eltype Int64:
 10
  0

deniedzeroixs(s::FlexTable)

Find the indices of subgraphs that have "deniedAt" equal to zero.

julia> using AllocationOpt
julia> using TheGraphData
julia> s = flextable([
           Dict("ipfsHash" => "Qma", "signalledTokens" => 5.0, "deniedAt" => 0),
           Dict("ipfsHash" => "Qmb", "signalledTokens" => 10.0, "deniedAt" => 10),
           Dict("ipfsHash" => "Qmc", "signalledTokens" => 15.0, "deniedAt" => 0),
       ])
julia> AllocationOpt.deniedzeroixs(s)
2-element Vector{Int64}:
 1
 3

dual(Ω, ψ, σ)

Analytic solution of the dual form of the optimisation problem given signals ψ, allocation vector Ω, and stake σ.

execute(
    a::FlexTable,
    ix::Integer,
    s::FlexTable,
    xs::AbstractMatrix{T},
    ps::AbstractMatrix{T},
    config::AbstractDict
) where {T<:Real}

Execute the actions picked by the optimiser.

julia> using AllocationOpt
julia> using TheGraphData
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa")
        ])
julia> xs = [[2.5 5.0]; [2.5 0.0]]
julia> ps = [[3.0 5.0]; [3.0 0.0]]
julia> s = flextable([
            Dict("stakedTokens" => "1", "signalledTokens" => "0", "ipfsHash" => "Qma"),
            Dict("stakedTokens" => "2", "signalledTokens" => "0", "ipfsHash" => "Qmb"),
        ])
julia> config = Dict("execution_mode" => "none")
julia> ix = 1
julia> AllocationOpt.execute(a, ix, s, xs, ps, config)

formatconfig!(config::AbstractDict)

Given a config, reformat values that need to be standardised.

julia> using AllocationOpt
julia> config = Dict("id" => "0xA")
julia> AllocationOpt.formatconfig!(config)
Dict{String, String} with 1 entry:
  "id" => "0xa"

frozen(a::FlexTable, config::AbstractDict)

The frozen stake of the indexer with allocations a.

julia> using AllocationOpt
julia> using TheGraphData
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "allocatedTokens" => 5),
            Dict("subgraphDeployment.ipfsHash" => "Qmb", "allocatedTokens" => 10),
       ])
julia> config = Dict("frozenlist" => ["Qma", "Qmb"])
julia> AllocationOpt.frozen(a, config)
15.0

groupunique(x::AbstractVector)

Find the indices of each unique value in x

julia> using AllocationOpt
julia> x = [1, 2, 1, 3, 2, 3]
julia> AllocationOpt.groupunique(x)
Dict{Vector{Int64}, Vector{Int64}} with 3 entries:
  [3] => [4, 6]
  [1] => [1, 3]
  [2] => [2, 5]

id(::Val{:allocation}, x)

Get the allocation id for each allocation in x.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict(
        "id" => "0x1"
    ),
])
julia> AllocationOpt.id(Val(:allocation), x)
1-element view(lazystack(::Vector{Vector{String}}), 1, :) with eltype String:
 "0x1"

indexingreward(x::Real, Ω::Real, ψ::Real, Φ::Real, Ψ::Real)

The indexing rewards for the allocation scalar x given signals ψ, the existing allocation on subgraphs Ω, token issuance Φ, and total signal Ψ.

julia> using AllocationOpt
julia> ψ = 0.0
julia> Ω = 1.0
julia> Φ = 1.0
julia> Ψ = 2.0
julia> x = 1.0
julia> AllocationOpt.indexingreward(x, Ω, ψ, Φ, Ψ)
0.0

indexingreward(
    ixs::AbstractArray{Integer},
    x::AbstractVector{Real},
    Ω::AbstractVector{Real},
    ψ::AbstractVector{Real},
    Φ::Real,
    Ψ::Real
)

The indexing rewards for the allocation vector x given signals ψ, the existing allocations on subgraphs Ω, token issuance Φ, and total signal Ψ. Here ixs is a vector of indices Ω, and ψ. x will be filtered by SemioticOpt, so we don't do this here.

julia julia> using AllocationOpt julia> ixs = Int32[2] julia> ψ = [0.0, 1.0] julia> Ω = [1.0, 1.0] julia> Φ = 1.0 julia> Ψ = 2.0 julia> x = [0.0, 1.0] julia> AllocationOpt.indexingreward(ixs, x, Ω, ψ, Φ, Ψ) 0.25`

indexingreward(
    x::AbstractVector{Real},
    Ω::AbstractVector{Real},
    ψ::AbstractVector{Real},
    Φ::Real,
    Ψ::Real
)

The indexing rewards for the allocation vector x given signals ψ, the existing allocations on subgraphs Ω, token issuance Φ, and total signal Ψ.

julia> using AllocationOpt
julia> ψ = [0.0, 1.0]
julia> Ω = [1.0, 1.0]
julia> Φ = 1.0
julia> Ψ = 2.0
julia> x = [0.0, 1.0]
julia> AllocationOpt.indexingreward(x, Ω, ψ, Φ, Ψ)
0.25

ipfshash(::Val{:allocation}, x)

Get the ipfs hash of x when x is part of the allocation table.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict(
        "subgraphDeployment.ipfsHash" => "Qma",
    ),
])
julia> AllocationOpt.ipfshash(Val(:allocation), x)
1-element view(lazystack(::Vector{Vector{String}}), 1, :) with eltype String:
 "Qma"

ipfshash(::Val{:subgraph}, x)

Get the ipfs hash of x when x is part of the allocation table.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict(
        "ipfsHash" => "Qma",
    ),
])
julia> AllocationOpt.ipfshash(Val(:subgraph), x)
1-element view(lazystack(::Vector{Vector{String}}), 1, :) with eltype String:
 "Qma"

iquery(id::AbstractString)

Return the components of a GraphQL query for the stake of indexer id.

For use with the TheGraphData.jl package.

julia> using AllocationOpt
julia> id = "0xa"
julia> value, args, fields = AllocationOpt.iquery(id)
("indexer", Dict{String, Union{Int64, Dict{String, String}, String}}("id" => "0xa"), ["delegatedTokens", "stakedTokens", "lockedTokens"])

Extended Help

You can find TheGraphData.jl at https://github.com/semiotic-ai/TheGraphData.jl

lipschitzconstant(ψ, Ω)

The Lipschitz constant of the indexing reward function given signals ψ and allocations Ω.

julia> using AllocationOpt
julia> ψ = [0.0, 1.0]
julia> Ω = [1.0, 1.0]
julia> AllocationOpt.lipschitzconstant(ψ, Ω)
2.0

locked(::Val{:indexer}, x)

The locked tokens of the indexer in table x.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict(
        "lockedTokens" => 10,
    ),
])
julia> AllocationOpt.locked(Val(:indexer), x)
10

newtokenissuance(n::FlexTable, config::Dict)

How many new tokens are issued over the allocation lifetime given network parameters n. Calcualted by networkGRTIssuancePerBlock * epochLength * allocation_lifetime

julia> using AllocationOpt
julia> using TheGraphData
julia> n = flextable([
            Dict(
                "id" => 1,
                "networkGRTIssuancePerBlock" => 2,
                "epochLength" => 1,
                "totalTokensSignalled" => 2,
                "currentEpoch" => 1,
            )
        ])
julia> config = Dict("allocation_lifetime" => 1)
julia> AllocationOpt.newtokenissuance(n, config)
1.0

nonzero(v::AbstractVector)

Get the non-zero elements of vector v.

julia> using AllocationOpt
julia> v = [0.0, 1.0]
julia> AllocationOpt.nonzero(v)
1-element view(::Vector{Float64}, [2]) with eltype Float64:
 1.0

nquery()

Return the components of a GraphQL query for network parameters.

For use with the TheGraphData.jl package.

julia> using AllocationOpt
julia> value, args, fields = AllocationOpt.nquery()
("graphNetwork", Dict("id" => 1), ["id", "networkGRTIssuance", "epochLength", "totalTokensSignalled", "currentEpoch"])

Extended Help

You can find TheGraphData.jl at https://github.com/semiotic-ai/TheGraphData.jl

optimize(Ω, ψ, σ, K, Φ, Ψ, g, rixs, config::AbstractDict)

Find the optimal solution vector given allocations of other indexers Ω, signals ψ, available stake σ, new tokens issued Φ, total signal Ψ, and gas in grt g. rixs are the indices of subgraphs that are eligible to receive indexing rewards.

Dispatches to optimize with the opt_mode key.

If opt_mode is fast, then run projected gradient descent with GSSP and Halpern. If opt_mode is optimal, then run Pairwise Greedy Optimisation.

julia> using AllocationOpt
julia> config = Dict("opt_mode" => "fast")
julia> rixs = [1, 2]
julia> Ω = [1.0, 1.0]
julia> ψ = [10.0, 10.0]
julia> σ = 5.0
julia> K = 2
julia> Φ = 1.0
julia> Ψ = 20.0
julia> g = 0.01
julia> xs, nonzeros, profits = AllocationOpt.optimize(Ω, ψ, σ, K, Φ, Ψ, g, rixs, config)
([5.0 2.5; 0.0 2.5], Int32[1, 2], [0.4066666666666667 0.34714285714285714; 0.0 0.34714285714285714])

optimize(::Val{:fast}, Ω, ψ, σ, K, Φ, Ψ, g, rixs)

Find the optimal vectors for k ∈ [1,K] given allocations of other indexers Ω, signals ψ, available stake σ, new tokens issued Φ, total signal Ψ, and gas in grt g. rixs are the indices of subgraphs that are eligible to receive indexing rewards.

julia> using AllocationOpt
julia> rixs = [1, 2]
julia> Ω = [1.0, 1.0]
julia> ψ = [10.0, 10.0]
julia> σ = 5.0
julia> K = 2
julia> Φ = 1.0
julia> Ψ = 20.0
julia> g = 0.01
julia> xs, nonzeros, profits = AllocationOpt.optimize(Val(:fast), Ω, ψ, σ, K, Φ, Ψ, g, rixs)
([5.0 2.5; 0.0 2.5], Int32[1, 2], [0.4066666666666667 0.34714285714285714; 0.0 0.34714285714285714])

optimize(::Val{:optimal}, Ω, ψ, σ, K, Φ, Ψ, g, rixs)

Find the optimal solution vector given allocations of other indexers Ω, signals ψ, available stake σ, new tokens issued Φ, total signal Ψ, and gas in grt g. rixs are the indices of subgraphs that are eligible to receive indexing rewards.

Example

julia> using AllocationOpt
julia> rixs = [1, 2]
julia> Ω = [1.0, 1.0]
julia> ψ = [10.0, 10.0]
julia> σ = 5.0
julia> K = 2
julia> Φ = 1.0
julia> Ψ = 20.0
julia> g = 0.01
julia> xs, nonzeros, profits = AllocationOpt.optimize(
           Val(:optimal), Ω, ψ, σ, K, Φ, Ψ, g, rixs
       )
([5.0 2.5; 0.0 2.5], Int32[1, 2], [0.4066666666666667 0.34714285714285714; 0.0 0.34714285714285714])

optimizeanalytic(Ω, ψ, σ)

Optimise analytically over existing allocation vector Ω, signals ψ, and stake σ.

julia> using AllocationOpt
julia> Ω = [1.0, 7.0]
julia> ψ = [10.0, 5.0]
julia> σ = 5.0
julia> AllocationOpt.optimizeanalytic(Ω, ψ, σ)
2-element Vector{Float64}:
 3.5283092056122474
 1.4716907943877526

optimizek(::Val{:fast}, x₀, Ω, ψ, σ, k, Φ, Ψ)

Find the optimal k sparse vector given initial value x₀, allocations of other indexers Ω, signals ψ, available stake σ, new tokens issued Φ, and total signal Ψ.

julia> using AllocationOpt
julia> x₀ = [2.5, 2.5]
julia> Ω = [1.0, 1.0]
julia> ψ = [10.0, 10.0]
julia> σ = 5.0
julia> k = 1
julia> Φ = 1.0
julia> Ψ = 20.0
julia> AllocationOpt.optimizek(Val(:fast), x₀, Ω, ψ, σ, k, Φ, Ψ)
2-element Vector{Float64}:
 5.0
 0.0

optimizek(::Val{:optimal}, x₀, Ω, ψ, σ, k, Φ, Ψ, g)

Find the optimal k sparse vector given allocations of other indexers Ω, signals ψ, available stake σ, new tokens issued Φ, total signal Ψ, and gas g.

Example

julia> using AllocationOpt
julia> Ω = [1.0, 1.0]
julia> ψ = [10.0, 10.0]
julia> σ = 5.0
julia> k = 1
julia> Φ = 1.0
julia> Ψ = 20.0
julia> g = 0.01
julia> x₀ = zeros(length(Ω))
julia> x = AllocationOpt.optimizek(Val(:optimal), x₀, Ω, ψ, σ, k, Φ, Ψ, g)
2-element Vector{Float64}:
 5.0
 0.0

pinned(config::AbstractDict)

The pinned vector of the indexer.

julia> using AllocationOpt
julia> s = flextable([
    Dict("ipfsHash" => "Qma", "signalledTokens" => 5.0),
    Dict("ipfsHash" => "Qmb", "signalledTokens" => 10.0),
    Dict("ipfsHash" => "Qmc", "signalledTokens" => 15.0),
])
julia> config = Dict("pinnedlist" => ["Qma", "Qmb"])
julia> AllocationOpt.pinned(s, config)
3-element Vector{Float64}:
 0.1
 0.1
 0.0

primal(Ω, ψ, ν)

Analytic solution of the primal form of the optimisation problem given signals ψ, allocations Ω, and a dual solution vector ν.

profit(r::Real, g::Real)

Compute the profit for one allocation with reward r and gas cost g.

julia> using AllocationOpt
julia> r = 10
julia> g = 1
julia> AllocationOpt.profit(r, g)
9

read(config::AbstractDict)

Given a config, read the data in as flextables.

If you have specified a "readdir" in the config, this will read from CSV files in that directory. Otherwise, this will query the specified "network_subgraph_endpoint"

julia> using AllocationOpt
julia> config = Dict("verbose" => false, "readdir" => "mydatadir")
julia> i, a, s, n = AllocationOpt.read(config)  # Read data from CSVs

julia> using AllocationOpt
julia> config = Dict(
    "verbose" => false,
    "network_subgraph_endpoint" => "https://api.thegraph.com/subgraphs/name/graphprotocol/graph-network-mainnet",
    "readdir" => nothing,
)
julia> i, a, s, n = AllocationOpt.read(config)  # Query GQL endpoint

read(f::AbstractString, config::AbstractDict)

Read the CSV files from f and return the tables from those files.

julia> using AllocationOpt
julia> i, a, s, n = AllocationOpt.read("myreaddir", Dict("verbose" => true))

read(::Nothing, config::AbstractDict)

Query the required data from the provided endpoint in the config.

julia> using AllocationOpt
julia> config = Dict(
            "verbose" => true,
            "network_subgraph_endpoint" => "https://api.thegraph.com/subgraphs/name/graphprotocol/graph-network-mainnet",
        )
julia> i, a, s, n = AllocationOpt.read(nothing, config)

readconfig(p::AbstractString)

Read the config file from path p. The config file must be specifed as a TOML.

See configuredefaults! to see which fields you should specify in the config.

julia> using AllocationOpt
julia> path = "myconfig.toml"
julia> config = AllocationOpt.readconfig(path)
Dict{String, Any} with 13 entries:
  "execution_mode"       => "none"
  "writedir"             => "data"
  "num_reported_options" => 2
  "id"                   => "0xd75c4dbcb215a6cf9097cfbcc70aab2596b96a9c"
  "pinnedlist"           => Union{}[]
  "gas"                  => 100
  "allocation_lifetime"  => 28
  "blacklist"            => Union{}[]
  "verbose"              => true
  "min_signal"           => 100
  "whitelist"            => Union{}[]
  "max_allocations"      => 5
  "frozenlist"           => Union{}[]
  "protocol_network"     => "mainnet"
  "syncing_networks"     => String["mainnet", "gnosis"]

reallocate_action(::Val{:actionqueue}, a::FlexTable, t::FlexTable, config::AbstractDict)

Create and push reallocate actions to the action queue.

julia> using AllocationOpt
julia> using TheGraphData
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa")
        ])
julia> t = flextable([
    Dict("amount" => "1", "profit" => "0", "ipfshash" => "Qma"),
    Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"),
])
julia> config = Dict("indexer_url" => "http://localhost:18000")
julia> TheGraphData.client!(config["indexer_url"])
julia> AllocationOpt.reallocate_action(Val(:actionqueue), a, t, config)
1-element Vector{Dict{String, Any}}:
 Dict("amount" => "1", "priority" => 0, "status" => AllocationOpt.queued, "allocationID" => "0xa", "source" => "AllocationOpt", "reason" => "Expected profit: 0", "type" => AllocationOpt.reallocate, "deploymentID" => "Qma", "protocolNetwork" => "mainnet")

reallocate_action(::Val{:none}, a, t, config)

Do nothing.

```julia julia> using AllocationOpt julia> using TheGraphData julia> a = flextable([ Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa") ]) julia> t = flextable([ Dict("amount" => "1", "profit" => "0", "ipfshash" => "Qma"), Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"), ]) julia> AllocationOpt.reallocate_action(Val(:none), a, t, Dict())

reallocate_action(::Val{:rules}, a::FlexTable, t::FlexTable, config::AbstractDict)

Print a rule that reallocates the old allocation with a new allocation amount

julia> using AllocationOpt
julia> using TheGraphData
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa")
        ])
julia> t = flextable([
    Dict("amount" => "1", "profit" => "0", "ipfshash" => "Qma"),
    Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"),
])
julia> AllocationOpt.reallocate_action(Val(:rules), a, t, Dict())
graph indexer rules stop Qma
Check allocation status being closed before submitting: graph indexer rules set Qma decisionBasis always allocationAmount 1
1-element Vector{String}:
 "graph indexer rules stop Qm" ⋯ 122 bytes ⋯ "asis always allocationAmount 1"

reportingtable(
    s::FlexTable, xs::AbstractMatrix{Real}, ps::AbstractMatrix{Real}, i::Integer
)

Construct a table for the strategy mapping the ipfshash, allocation amount, and profit

julia> using AllocationOpt
julia> s = flextable([
        Dict("stakedTokens" => "1", "signalledTokens" => "2", "ipfsHash" => "Qma"),
        Dict("stakedTokens" => "2", "signalledTokens" => "1", "ipfsHash" => "Qmb"),
    ])
julia> xs = [[2.5 5.0]; [2.5 0.0]]
julia> ps = [[3.0 5.0]; [3.0 0.0]]
julia> i = 1
julia> AllocationOpt.reportingtable(s, xs, ps, i)
FlexTable with 3 columns and 2 rows:
     ipfshash  amount  profit
   ┌─────────────────────────
 1 │ Qma       2.5     3.0
 2 │ Qmb       2.5     3.0

savenames(p::AbstractString)

Return a generator of the generic names of the CSV files containing the data with the path specified by p.

julia> using AllocationOpt
julia> path = "mypath"
julia> paths = AllocationOpt.savenames(path)
Base.Generator{NTuple{4, String}, AllocationOpt.var"#1#2"{String}}(AllocationOpt.var"#1#2"{String}("mypath"), ("indexer.csv", "allocation.csv", "subgraph.csv", "network.csv"))

signal(::Val{:network}, x)

The total signal in the network

```julia julia> using AllocationOpt julia> using TheGraphData julia> n = flextable([ Dict( "id" => 1, "networkGRTIssuancePerBlock" => 1, "epochLength" => 28, "totalTokensSignalled" => 2, "currentEpoch" => 1, ) ]) julia> AllocationOpt.signal(Val(:network), n) 2

signal(::Val{:subgraph}, x)

The tokens signalled on the subgraphs in table x.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict("signalledTokens" => 10,),
    Dict("signalledTokens" => 5,),
])
julia> AllocationOpt.signal(Val(:subgraph), x)
2-element view(transpose(lazystack(::Vector{Vector{Int64}})), :, 1) with eltype Int64:
 10
  5

sortprofits!(NamedTuple{Tuple{Float64, Int64}})

Sort the nonzero best profits from highest to lowest

julia> using AllocationOpt
julia> popts = [
        (; :profit => 5.0, :index => 2),
        (; :profit => 6.0, :index => 1)
    ]
julia> popts = AllocationOpt.sortprofits!(popts)
2-element Vector{NamedTuple{(:profit, :index), Tuple{Float64, Int64}}}:
 (profit = 6.0, index = 1)
 (profit = 5.0, index = 2)

squery(config::AbstractDict)

Return the components of a GraphQL query for subgraphs.

For use with the TheGraphData.jl package.

julia> using AllocationOpt
julia> config = Dict("syncing_networking" => ["mainnet"])
julia> value, args, fields = AllocationOpt.squery(config)
("subgraphDeployments", Dict{String, Union{Dict{String, String}, String}}(), ["ipfsHash", "signalledTokens", "stakedTokens"])

Extended Help

You can find TheGraphData.jl at https://github.com/semiotic-ai/TheGraphData.jl

stake(::Val{:allocation}, x)

Get the allocated tokens for each allocation in x.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict(
        "allocatedTokens" => 1,
    ),
])
julia> AllocationOpt.stake(Val(:allocation), x)
1-element view(transpose(lazystack(::Vector{Vector{Int64}})), :, 1) with eltype Int64:
 1

stake(::Val{:indexer}, x)

The tokens staked by the indexer in table x.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict(
        "stakedTokens" => 10,
    ),
])
julia> AllocationOpt.stake(Val(:indexer), x)
10

stake(::Val{:subgraph}, x)

The tokens staked on the subgraphs in table x.

julia> using AllocationOpt
julia> using TheGraphData
julia> x = flextable([
    Dict("stakedTokens" => 10,),
    Dict("stakedTokens" => 5,),
])
julia> AllocationOpt.stake(Val(:subgraph), x)
2-element view(transpose(lazystack(::Vector{Vector{Int64}})), :, 1) with eltype Int64:
 10
  5

strategydict(
    p::NamedTuple,
    xs::AbstractMatrix{Real},
    nonzeros::AbstractVector{Integer},
    fs::FlexTable,
    profitmatrix::AbstractMatrix{Real}
)

For a profit, index pair p, generate the nested dictionary representing the data to convert to a JSON string. xs is the allocation strategy matrix, nonzeros are the number of nonzeros in each allocation strategy, fs is a table containing subgraph ipfshashes, and the profitmatrix is a matrix containing profit for each allocation in xs

julia> using AllocationOpt
julia> using TheGraphData
julia> popts = [
        (; :profit => 6.0, :index => 1),
        (; :profit => 5.0, :index => 2)
    ]
julia> xs = [[2.5 5.0]; [2.5 0.0]]
julia> profits = [[3.0 5.0]; [3.0 0.0]]
julia> nonzeros = [2, 1]
julia> fs = flextable([
        Dict("stakedTokens" => "1", "signalledTokens" => "0", "ipfsHash" => "Qma"),
        Dict("stakedTokens" => "2", "signalledTokens" => "0", "ipfsHash" => "Qmb"),
    ])
julia> AllocationOpt.strategydict.(popts, Ref(xs), Ref(nonzeros), Ref(fs), Ref(profits))
2-element Vector{Dict{String, Any}}:
 Dict("num_allocations" => 2, "profit" => 6.0, "allocations" => Dict{String, Any}[Dict("allocationAmount" => "2.5", "profit" => 3.0, "deploymentID" => "Qma"), Dict("allocationAmount" => "2.5", "profit" => 3.0, "deploymentID" => "Qmb")])
 Dict("num_allocations" => 1, "profit" => 5.0, "allocations" => Dict{String, Any}[Dict("allocationAmount" => "5", "profit" => 5.0, "deploymentID" => "Qma")])

subtractindexer!(a::FlexTable, s::FlexTable)

Subtract the indexer's allocated tokens from the total allocated tokens on each subgraph.

julia> using AllocationOpt
julia> using TheGraphData
julia> s = flextable([
            Dict("ipfsHash" => "Qmb", "stakedTokens" => 20),
            Dict("ipfsHash" => "Qma", "stakedTokens" => 10),
            Dict("ipfsHash" => "Qmc", "stakedTokens" => 5),
        ])
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "allocatedTokens" => 5, "id" => "0xa"),
            Dict("subgraphDeployment.ipfsHash" => "Qmb", "allocatedTokens" => 10, "id" => "0xb"),
        ])
julia> a, s = AllocationOpt.subtractindexer!(a, s)
(NamedTuple[(var"subgraphDeployment.ipfsHash" = "Qma", allocatedTokens = 5, id = "0xa"), (var"subgraphDeployment.ipfsHash" = "Qmb", allocatedTokens = 10, id = "0xb")], NamedTuple[(stakedTokens = 5.0, ipfsHash = "Qma"), (stakedTokens = 10, ipfsHash = "Qmb"), (stakedTokens = 5, ipfsHash = "Qmc")])

togrt(x::AbstractString)

Convert x to GRT.

julia> using AllocationOpt
julia> AllocationOpt.togrt("1")
1.0e-18

unallocate_action(::Val{:actionqueue}, a::FlexTable, t::FlexTable, config::AbstractDict)

Create and push the unallocate actions to the action queue.

```julia julia> using AllocationOpt julia> using TheGraphData julia> a = flextable([ Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa") ]) julia> t = flextable([ Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"), ]) julia> config = Dict( "frozenlist" => [], "indexerurl" => "http://localhost:18000" ) julia> TheGraphData.client!(config["indexerurl"]) julia> AllocationOpt.unallocate_action(Val(:actionqueue), a, t, config) 1-element Vector{Dict{String, Any}}: Dict("priority" => 0, "status" => AllocationOpt.queued, "allocationID" => "0xa", "source" => "AllocationOpt", "reason" => "AllocationOpt", "type" => AllocationOpt.unallocate, "deploymentID" => "Qma", "protocolNetwork" => "mainnet")

unallocate_action(::Val{:none}, a, t, config)

Do nothing.

julia> using AllocationOpt
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa")
        ])
julia> t = flextable([
            Dict("amount" => "1", "profit" => "0", "ipfshash" => "Qma"),
            Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"),
        ])
julia> AllocationOpt.unallocate_action(Val(:none), a, t, Dict())

unallocate_action(::Val{:rules}, a::FlexTable, t::FlexTable, config::AbstractDict)

Print a rule that stops old allocations that the optimiser has not chosen and that aren't frozen.

julia> using AllocationOpt
julia> a = flextable([
            Dict("subgraphDeployment.ipfsHash" => "Qma", "id" => "0xa")
        ])
julia> t = flextable([
            Dict("amount" => "2", "profit" => "0", "ipfshash" => "Qmb"),
        ])
julia> AllocationOpt.unallocate_action(Val(:rules), a, t, Dict("frozenlist" => []))
graph indexer rules stop Qma
1-element Vector{String}:
 "graph indexer rules stop Qma"

write(i::FlexTable, a::FlexTable, s::FlexTable, n::FlexTable, config::AbstractDict)

Write the tables to the writedir specified in the config.

julia> using AllocationOpt
julia> using TheGraphData
julia> config = Dict("verbose" => true, "writedir" => "datadir")
julia> t = flextable([
            Dict("ipfsHash" => "Qma", "signalledTokens" => "1"),
            Dict("ipfsHash" => "Qmb", "signalledTokens" => "2"),
        ])
julia> i, a, s, n = repeat([t,], 4)
juila> AllocationOpt.write(i, a, s, n, config)

function writejson(results::AbstractString, config::AbstractDict)

Write the optimized results to the writedir specified in the config.

julia> Using AllocationOpt
julia> results = "{"strategies":[{"num_allocations":2,"profit":6.0,"allocations":[{"allocationAmount":2.5,"profit":3.0,"deploymentID":"Qma"},{"allocationAmount":2.5,"profit":3.0,"deploymentID":"Qmb"}]},{"num_allocations":1,"profit":5.0,"allocations":[{"allocationAmount":5.0,"profit":5.0,"deploymentID":"Qma"}]}]}"
julia> config = Dict{"writedir" => "."}
julia> AllocationOpt.writejson(results, config)

iteration(f::Function, a::AnalyticOpt)

Perform the analytic optimisation.

x!(a::AnalyticOpt, v)

In-place setting of a.x to v

See SemioticOpt.x.

x(a::AnalyticOpt)

Return the current best guess for the solution.