Binary Data Transfer

Learn how to transfer images, files, and other binary data between Flutter and Unity with support for chunking, compression, and performance optimization.

Overview

The BinaryDataExample.cs template demonstrates:

Receiving binary data (images, files) from Flutter
Sending binary data (screenshots, textures) to Flutter
Chunked transfer for large files
Compression support
Error handling and validation

Binary messaging is essential for transferring images, audio files, save data, and other non-text content.

Setup

Sync Templates

game sync scripts --templates

Unity Setup

Create an empty GameObject named BinaryData
Attach BinaryDataExample.cs script
Configure settings in Inspector:
- Max Texture Size: 2048
- Enable Compression: true

Receiving Binary Data

Flutter → Unity (Images)

Flutter Side

import 'dart:io';
import 'package:flutter/material.dart';
import 'package:image_picker/image_picker.dart';
import 'package:gameframework/gameframework.dart';

class BinaryDataExample extends StatefulWidget {
  @override
  State<BinaryDataExample> createState() => _BinaryDataExampleState();
}

class _BinaryDataExampleState extends State<BinaryDataExample> {
  GameEngineController? _controller;
  
  Future<void> _pickAndSendImage() async {
    final picker = ImagePicker();
    final image = await picker.pickImage(source: ImageSource.gallery);
    
    if (image != null) {
      final bytes = await image.readAsBytes();
      
      // Send image to Unity
      await _controller?.sendBinaryMessage(
        'BinaryData',
        'receiveImage',
        bytes,
      );
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Binary Data Transfer')),
      body: Column(
        children: [
          Expanded(
            child: GameWidget(
              engineType: GameEngineType.unity,
              config: GameEngineConfig(runImmediately: true),
              onEngineCreated: (controller) {
                setState(() => _controller = controller);
              },
              onMessage: _onMessage,
            ),
          ),
          Padding(
            padding: EdgeInsets.all(16),
            child: ElevatedButton.icon(
              onPressed: _pickAndSendImage,
              icon: Icon(Icons.image),
              label: Text('Send Image to Unity'),
            ),
          ),
        ],
      ),
    );
  }

  void _onMessage(GameEngineMessage message) {
    if (message.method == 'onImageReceived') {
      final data = jsonDecode(message.data);
      if (data['success']) {
        print('Image loaded: ${data['width']}x${data['height']}');
      } else {
        print('Error: ${data['error']}');
      }
    }
  }
}

Unity Side

using UnityEngine;
using Xraph.GameFramework.Unity;

public class BinaryDataExample : FlutterMonoBehaviour
{
    protected override string TargetName => "BinaryData";

    [SerializeField] private int maxTextureSize = 2048;
    private Texture2D _lastReceivedTexture;

    [FlutterMethod("receiveImage", AcceptsBinary = true)]
    public void ReceiveImage(byte[] imageData)
    {
        Debug.Log($"Received image: {imageData.Length} bytes");

        try
        {
            // Create texture from image data
            _lastReceivedTexture = new Texture2D(2, 2);
            bool loaded = _lastReceivedTexture.LoadImage(imageData);

            if (loaded)
            {
                Debug.Log($"Texture: {_lastReceivedTexture.width}x{_lastReceivedTexture.height}");

                SendToFlutter("onImageReceived", new ImageReceivedEvent
                {
                    success = true,
                    width = _lastReceivedTexture.width,
                    height = _lastReceivedTexture.height,
                    format = _lastReceivedTexture.format.ToString()
                });

                // Apply texture to material
                ApplyTexture(_lastReceivedTexture);
            }
            else
            {
                SendToFlutter("onImageReceived", new ImageReceivedEvent
                {
                    success = false,
                    error = "Failed to load image data"
                });
            }
        }
        catch (System.Exception e)
        {
            Debug.LogError($"Error loading image: {e.Message}");
            SendToFlutter("onImageReceived", new ImageReceivedEvent
            {
                success = false,
                error = e.Message
            });
        }
    }

    private void ApplyTexture(Texture2D texture)
    {
        // Apply to a material in your scene
        var renderer = GetComponent<Renderer>();
        if (renderer != null)
        {
            renderer.material.mainTexture = texture;
        }
    }
}

[System.Serializable]
public class ImageReceivedEvent
{
    public bool success;
    public int width;
    public int height;
    public string format;
    public string error;
}

Sending Binary Data

Unity → Flutter (Screenshots)

Unity Side

[FlutterMethod("requestScreenshot")]
public void RequestScreenshot(ScreenshotRequest request)
{
    StartCoroutine(CaptureScreenshot(request.width, request.height));
}

private IEnumerator CaptureScreenshot(int width, int height)
{
    // Wait for end of frame
    yield return new WaitForEndOfFrame();

    // Capture screenshot
    RenderTexture rt = RenderTexture.GetTemporary(width, height);
    Camera.main.targetTexture = rt;
    Camera.main.Render();

    Texture2D screenshot = new Texture2D(width, height, TextureFormat.RGB24, false);
    RenderTexture.active = rt;
    screenshot.ReadPixels(new Rect(0, 0, width, height), 0, 0);
    screenshot.Apply();

    Camera.main.targetTexture = null;
    RenderTexture.active = null;
    RenderTexture.ReleaseTemporary(rt);

    // Encode to PNG
    byte[] bytes = screenshot.EncodeToPNG();
    
    // Send to Flutter
    SendBinaryToFlutter("onScreenshot", bytes);
    
    Destroy(screenshot);
}

Flutter Side

import 'dart:convert';
import 'dart:typed_data';
import 'package:flutter/material.dart';

class ScreenshotViewer extends StatefulWidget {
  @override
  State<ScreenshotViewer> createState() => _ScreenshotViewerState();
}

class _ScreenshotViewerState extends State<ScreenshotViewer> {
  GameEngineController? _controller;
  Uint8List? _screenshotBytes;

  void _requestScreenshot() {
    _controller?.sendJsonMessage('BinaryData', 'requestScreenshot', {
      'width': 1920,
      'height': 1080,
    });
  }

  void _onMessage(GameEngineMessage message) {
    if (message.method == 'onScreenshot') {
      // Parse binary data from message
      final data = jsonDecode(message.data);
      final bytes = base64Decode(data['data']);
      
      setState(() {
        _screenshotBytes = bytes;
      });
    }
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children: [
        ElevatedButton(
          onPressed: _requestScreenshot,
          child: Text('Capture Screenshot'),
        ),
        if (_screenshotBytes != null)
          Expanded(
            child: Image.memory(_screenshotBytes!),
          ),
      ],
    );
  }
}

Chunked Transfer

For large files, use chunked transfer:

Flutter Side

Future<void> _sendLargeFile(File file) async {
  final bytes = await file.readAsBytes();
  const chunkSize = 1024 * 1024; // 1MB chunks
  final totalChunks = (bytes.length / chunkSize).ceil();

  for (int i = 0; i < totalChunks; i++) {
    final start = i * chunkSize;
    final end = min((i + 1) * chunkSize, bytes.length);
    final chunk = bytes.sublist(start, end);

    await _controller?.sendJsonMessage('BinaryData', 'receiveChunk', {
      'chunkIndex': i,
      'totalChunks': totalChunks,
      'data': base64Encode(chunk),
    });
  }
}

Unity Side

private Dictionary<string, ChunkedTransfer> _activeTransfers = 
    new Dictionary<string, ChunkedTransfer>();

[FlutterMethod("receiveChunk")]
public void ReceiveChunk(ChunkMessage message)
{
    string transferId = message.transferId ?? "default";

    if (!_activeTransfers.ContainsKey(transferId))
    {
        _activeTransfers[transferId] = new ChunkedTransfer
        {
            totalChunks = message.totalChunks,
            chunks = new byte[message.totalChunks][]
        };
    }

    var transfer = _activeTransfers[transferId];
    transfer.chunks[message.chunkIndex] = Convert.FromBase64String(message.data);
    transfer.receivedChunks++;

    // Send progress update
    SendToFlutter("onTransferProgress", new TransferProgress
    {
        transferId = transferId,
        progress = (float)transfer.receivedChunks / transfer.totalChunks
    });

    // Check if complete
    if (transfer.receivedChunks == transfer.totalChunks)
    {
        // Combine chunks
        byte[] completeData = CombineChunks(transfer.chunks);
        OnTransferComplete(transferId, completeData);
        _activeTransfers.Remove(transferId);
    }
}

Compression

Unity Side - Compress Before Sending

using System.IO;
using System.IO.Compression;

private byte[] CompressData(byte[] data)
{
    using (var output = new MemoryStream())
    {
        using (var gzip = new GZipStream(output, CompressionMode.Compress))
        {
            gzip.Write(data, 0, data.Length);
        }
        return output.ToArray();
    }
}

[FlutterMethod("requestCompressedData")]
public void RequestCompressedData()
{
    byte[] data = GetLargeData();
    byte[] compressed = CompressData(data);
    
    Debug.Log($"Original: {data.Length} bytes, Compressed: {compressed.Length} bytes");
    
    SendBinaryToFlutter("onCompressedData", compressed);
}

Flutter Side - Decompress

import 'dart:io';

Uint8List decompressData(Uint8List compressed) {
  return GZipCodec().decode(compressed);
}

void _onMessage(GameEngineMessage message) {
  if (message.method == 'onCompressedData') {
    final data = jsonDecode(message.data);
    final compressed = base64Decode(data['data']);
    final decompressed = decompressData(compressed);
    
    print('Received: ${compressed.length} bytes');
    print('Decompressed: ${decompressed.length} bytes');
  }
}

Performance Tips

1. Format Selection

Choose appropriate image formats:

// For screenshots - PNG (lossless)
byte[] pngData = texture.EncodeToPNG();

// For photos - JPEG (smaller)
byte[] jpgData = texture.EncodeToJPG(85); // 85% quality

// For textures - optimized formats
byte[] data = texture.GetRawTextureData();

2. Resize Before Sending

private Texture2D ResizeTexture(Texture2D source, int maxSize)
{
    if (source.width <= maxSize && source.height <= maxSize)
        return source;

    float ratio = Mathf.Min(
        (float)maxSize / source.width,
        (float)maxSize / source.height
    );

    int newWidth = Mathf.RoundToInt(source.width * ratio);
    int newHeight = Mathf.RoundToInt(source.height * ratio);

    return ResizeTexture(source, newWidth, newHeight);
}

3. Async Operations

Future<void> _sendImageAsync(File file) async {
  // Read in background
  final bytes = await compute(_readFile, file.path);
  
  // Send to Unity
  await _controller?.sendBinaryMessage('BinaryData', 'receiveImage', bytes);
}

static Uint8List _readFile(String path) {
  return File(path).readAsBytesSync();
}

Error Handling

[FlutterMethod("receiveData", AcceptsBinary = true)]
public void ReceiveData(byte[] data)
{
    if (data == null || data.Length == 0)
    {
        SendToFlutter("onError", new ErrorMessage
        {
            error = "No data received"
        });
        return;
    }

    if (data.Length > maxDataSize)
    {
        SendToFlutter("onError", new ErrorMessage
        {
            error = $"Data too large: {data.Length} bytes (max: {maxDataSize})"
        });
        return;
    }

    try
    {
        ProcessData(data);
        SendToFlutter("onSuccess", new { size = data.Length });
    }
    catch (Exception e)
    {
        SendToFlutter("onError", new ErrorMessage
        {
            error = e.Message
        });
    }
}

Always validate binary data size and format before processing to prevent memory issues.

Use Cases

Save Game Data

// Save game state
final saveData = jsonEncode(gameState);
final bytes = utf8.encode(saveData);
await controller.sendBinaryMessage('GameManager', 'saveGame', bytes);

Asset Streaming

// Download and send asset to Unity
final response = await http.get(Uri.parse(assetUrl));
await controller.sendBinaryMessage('AssetManager', 'loadAsset', response.bodyBytes);

User-Generated Content

// User uploads custom texture
final image = await ImagePicker().pickImage(source: ImageSource.gallery);
final bytes = await image!.readAsBytes();
await controller.sendBinaryMessage('CustomContent', 'uploadTexture', bytes);

What You'll Learn

✅ Sending images from Flutter to Unity
✅ Capturing screenshots in Unity
✅ Chunked file transfer
✅ Data compression
✅ Performance optimization
✅ Error handling

Binary messaging opens up possibilities for dynamic content, user uploads, and asset streaming!

Next Steps

High-Frequency Messaging

Optimize messaging performance

State Synchronization

Efficient state management

Messaging Patterns

Back to messaging basics

High-Frequency Messaging

State Synchronization

Messaging Patterns

On this page