Async Functions

Overview

WeaveFFI exposes asynchronous Rust operations through a single callback-based C ABI and language-native async wrappers in every target. Mark a function with async: true (and optionally cancellable: true) in the IDL and the generators emit the right shape per target: async in Swift (async throws when the function also declares throws: true), suspend fun in Kotlin, Promise<T> in JS, async def in Python, Task<T> in .NET, and so on. When an async function declares throws: true, the failure that settles the future is the module’s typed domain error (see the Error Handling Guide).

The completion contract every wrapper implements is stated once, in weaveffi_core::plan::AsyncProtocol: the callback fires exactly once per launch, from an arbitrary producer thread, and the result it receives is either borrowed (copy it inside the callback) or adopted (own it and destroy it later). See Result ownership and threading below.

When to use

Use async functions for:

• I/O-bound work (network, disk, database).
• Long-running operations that should not block the consumer’s event loop (UI threads, JS event loop, asyncio loop).
• Operations the consumer should be able to cancel (combine with cancellable: true).

Avoid async for:

• Short CPU-bound work (math, parsing, validation). The callback overhead is more expensive than the call itself.
• Functions whose Rust implementation is purely synchronous and finishes in microseconds.

Step-by-step

1. Declare the function in the IDL

version: "0.5.0"
modules:
  - name: net
    errors:
      name: NetError
      codes:
        - { name: Unreachable, code: 1, message: "host unreachable" }
    functions:
      - name: fetch_data
        params:
          - { name: url, type: string }
        return: string
        async: true
        throws: true
        doc: "Fetches data from the given URL"

      - name: upload_file
        params:
          - { name: path, type: string }
          - { name: data, type: bytes }
        return: bool
        async: true
        cancellable: true
        doc: "Uploads a file, can be cancelled"

Here fetch_data fails with a typed NetError, while upload_file is non-throwing: apart from cancellation, the only failures it can surface are producer bugs.

2. Implement it in Rust

The generated C ABI symbol takes a callback pointer and an opaque void* context. The Rust worker invokes the callback exactly once when it is done. With the #[weaveffi::module] macro you write a plain async fn (see samples/async-demo/src/lib.rs) and the launcher below is generated for you; a hand-written producer implements the same pattern:

#![allow(unused)]
#![allow(unsafe_code)]
#![allow(non_camel_case_types)]
#![allow(clippy::not_unsafe_ptr_arg_deref)]

fn main() {
use std::ffi::c_void;
use std::os::raw::c_char;
use weaveffi_abi::{self as abi, weaveffi_error};

pub type weaveffi_net_fetch_data_callback =
    extern "C" fn(context: *mut c_void, err: *mut weaveffi_error, result: *const c_char);

#[no_mangle]
pub extern "C" fn weaveffi_net_fetch_data_async(
    url: *const c_char,
    callback: weaveffi_net_fetch_data_callback,
    context: *mut c_void,
) {
    let url_str = abi::c_ptr_to_string(url).unwrap_or_default();
    let ctx = context as usize;
    std::thread::spawn(move || {
        let payload = abi::string_to_c_ptr(&format!("payload from {url_str}"));
        callback(ctx as *mut c_void, std::ptr::null_mut(), payload);
        // The string is borrowed by the callback: the producer frees it
        // after the callback returns, so the consumer must have copied.
        abi::free_string(payload);
    });
}
}

The async launcher symbol always carries the _async suffix (weaveffi_net_fetch_data_async), keeping the name free for a possible synchronous variant. Note who frees the result: buffer results (strings, byte arrays, lists, boxed optional scalars) are owned by the producer, which releases them after the callback returns; the macro-generated launchers do exactly this. Owned-object results (records, rich enums, interfaces) are the exception: the callback receives ownership of the pointer. See Result ownership and threading.

3. Call it from each target

Swift:

let payload = try await Net.fetchData("https://example.com/data")

Kotlin/Android:

val payload = Net.fetchData("https://example.com/data")

Node.js:

const payload = await fetchData("https://example.com/data");

Python:

payload = await fetch_data("https://example.com/data")

.NET:

var payload = await Net.FetchDataAsync("https://example.com/data");

Dart:

final payload = await fetchData('https://example.com/data');

Because fetch_data declares throws: true, the error that rejects the promise (or resumes the continuation, or fails the task) is the typed NetError, so a Swift consumer writes catch NetError.unreachable and a Python consumer writes except NetError.Unreachable.

4. Cancel a running operation

For cancellable: true functions the C launcher gains a weaveffi_cancel_token* slot (before callback and context), and the weaveffi-abi runtime provides the token lifecycle:

weaveffi_cancel_token* token = weaveffi_cancel_token_create();
weaveffi_net_upload_file_async(path, data, data_len, token, on_done, ctx);
/* later, from any thread: */
weaveffi_cancel_token_cancel(token);

The Rust worker polls weaveffi_cancel_token_is_cancelled(token) and stops early, but the callback is always invoked exactly once: either with the result or with a Cancelled error. The pin/unpin pair (see Reference) runs on the cancellation path identically to the success path.

Today the C and C++ surfaces expose the token (C++ as a trailing cancel_token = nullptr parameter); the other wrappers pass NULL. The operation runs to completion even if the consumer-side future is abandoned.

Reference

C ABI shape

Each async function gets its own callback typedef of the form (context, err, <result slots>), and a launcher with the _async suffix:

typedef void (*weaveffi_net_fetch_data_callback)(
    void* context,
    weaveffi_error* err,
    const char* result);

void weaveffi_net_fetch_data_async(
    const char* url,
    weaveffi_net_fetch_data_callback callback,
    void* context);

The err argument of the callback carries the domain code for a throws: true function; on a non-throwing function a non-zero code only ever reports a producer bug (see Throws versus Trap).

For cancellable: true the launcher takes a token slot before the callback, and the runtime provides the token lifecycle:

void weaveffi_net_upload_file_async(
    const char* path,
    const uint8_t* data, size_t data_len,
    weaveffi_cancel_token* cancel_token,
    weaveffi_net_upload_file_callback callback,
    void* context);

weaveffi_cancel_token* weaveffi_cancel_token_create(void);
void weaveffi_cancel_token_cancel(weaveffi_cancel_token* token);
bool weaveffi_cancel_token_is_cancelled(const weaveffi_cancel_token* token);
void weaveffi_cancel_token_destroy(weaveffi_cancel_token* token);

Result ownership and threading

The completion contract has three clauses, stated once in weaveffi_core::plan::AsyncProtocol and rendered by every wrapper:

1. Single completion. The callback fires exactly once per launch. The wrapper resolves its native future idiom (a Python asyncio future, a JS Promise, a Swift continuation, a C# TaskCompletionSource, a Go channel) exactly once and then releases the registration.
2. Borrowed results. Result buffers passed to the callback (strings, bytes, arrays, boxed optional scalars) are owned by the producer and valid only for the callback’s duration: the wrapper deep-copies them before the callback returns and must not free them. The producer releases them after the callback returns; the macro-generated launchers do this for you. Owned-object results (records, rich enums, and interfaces, including optionals of them) are the exception: the callback receives ownership and adopts the pointer into the wrapper’s disposal idiom, which eventually calls the type’s _destroy symbol.
3. Foreign-thread delivery. The callback runs on an arbitrary producer thread, so the wrapper hops back to its native scheduler before touching consumer state (Python’s call_soon_threadsafe, Node’s thread-safe function, a dispatched Swift continuation) rather than resolving inline where the target’s runtime forbids it.

The error struct passed to the callback is also producer-owned and borrowed for the callback’s duration: the wrapper copies the code and message inside the callback, and the producer releases the message afterward. A wrapper may also call weaveffi_error_clear itself; the clear is idempotent (it nulls the message pointer), so the producer’s own release stays safe.

If you consume the raw C surface directly, the same rules apply to your callback: copy every buffer before returning, adopt object pointers, and never free a borrowed result.

Per-target async surface

A wrapper that does not expose the token still launches and completes the call correctly; the operation simply runs to completion even if the consumer abandons the future. Drop to the C surface when you need cooperative cancellation from one of those targets.

Pin / unpin matrix

Every binding pins the user-supplied void* context and the callback closure for the lifetime of the operation, then releases them exactly once on the callback path. The matrix below is the contract every generator implements; each row is asserted by that generator’s unit tests.

Audit invariants

For every async-capable target:

1. The void* context has exactly one owner at any moment.
2. The callback closure is pinned by an explicit “+1” allocation (GCHandle.Alloc, Unmanaged.passRetained, NewGlobalRef, NativeCallable.listener, …) before the C worker can see it, and released by the matching “-1” exactly once on the callback path.
3. Synchronous failure of the C call (the callback never fires) is handled in a catch / try that frees the pin so it does not leak.
4. The async-demo sample exports weaveffi_tasks_active_callbacks() so a harness can assert the count returns to zero after a burst of concurrent calls.

Pitfalls

• Async void functions: the validator emits a warning. They are valid but almost always indicate a missing return type.
• Forgetting cancellable: true: without it, the launcher has no cancel-token slot and the operation cannot be cancelled at all.
• Using async for CPU-bound work: the callback overhead exceeds the work being done; keep it synchronous.
• Calling Go/Ruby async functions on a latency-sensitive thread: both wrappers block the calling thread until the producer completes. Wrap the call in a goroutine / Ruby Thread when you need concurrency; the native work already runs off-thread.
• Letting the callback closure get garbage-collected: every generator pins it explicitly. Do not strip those pins when editing generated code by hand.
• Returning null instead of invoking the callback: the contract is that the callback fires exactly once for every async call, including on cancellation.
• Holding a result pointer past the callback: buffer results are producer-owned and freed as soon as the callback returns. Copy the data inside the callback; a stashed pointer dangles.
• Freeing a borrowed result inside the callback: strings, bytes, and array buffers belong to the producer, which frees them itself. Freeing them in the callback double-frees. The only pointers the callback owns are object results (records, rich enums, interfaces), which it must eventually _destroy exactly once.