ByteArrayAsyncExtensions

This documentation covers asynchronous operations for byte arrays in ByteArrayAsyncExtensions. These methods provide non-blocking I/O operations, parallel processing capabilities, and async-friendly byte array manipulations.

Overview

ByteArrayAsyncExtensions offers comprehensive async support for:

• File I/O operations - Async reading and writing to files
• Stream operations - Non-blocking stream interactions
• Parallel processing - CPU-intensive operations with concurrency control
• Cryptographic operations - Async hashing and integrity verification
• Utility operations - Async searching and comparison
• Cancellation support - Proper cancellation token handling

Async File Operations

WriteToFileAsync

WriteToFileAsync

Asynchronously writes byte array data to a file without blocking the calling thread.

Usage Examples

// Basic async file writing
byte[] imageData = GetImageData();
string filePath = "output.jpg";

await imageData.WriteToFileAsync(filePath);
Console.WriteLine($"Wrote {imageData.Length} bytes to {filePath}");

// With cancellation support
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30));
try
{
    await largeData.WriteToFileAsync("large-file.bin", cts.Token);
}
catch (OperationCanceledException)
{
    Console.WriteLine("File write operation was cancelled");
}

// Web application file upload handling
[HttpPost]
public async Task<IActionResult> UploadFile(IFormFile file)
{
    if (file.Length > 0)
    {
        using var memoryStream = new MemoryStream();
        await file.CopyToAsync(memoryStream);
        byte[] fileData = memoryStream.ToArray();

        string fileName = Path.Combine("uploads", file.FileName);
        await fileData.WriteToFileAsync(fileName);

        return Ok($"File uploaded: {file.FileName}");
    }

    return BadRequest("No file uploaded");
}

// Batch file processing
var fileTasks = new List<Task>();
for (int i = 0; i < documentData.Count; i++)
{
    var data = documentData[i];
    var fileName = $"document_{i:000}.pdf";
    fileTasks.Add(data.WriteToFileAsync(fileName));
}

await Task.WhenAll(fileTasks);
Console.WriteLine($"Processed {fileTasks.Count} files");

// Error handling
try
{
    await sensitiveData.WriteToFileAsync("/protected/data.bin");
}
catch (UnauthorizedAccessException)
{
    Console.WriteLine("Access denied to write file");
}
catch (IOException ex)
{
    Console.WriteLine($"IO error: {ex.Message}");
}

ReadFromFileAsync

ReadFromFileAsync

Asynchronously reads file content into a byte array.

Usage Examples

// Basic async file reading
string configPath = "config.bin";
byte[] configData = await ByteArrayAsyncExtensions.ReadFromFileAsync(configPath);
Console.WriteLine($"Read {configData.Length} bytes from config");

// With cancellation and timeout
using var cts = new CancellationTokenSource(TimeSpan.FromMinutes(5));
try
{
    byte[] largeFile = await ByteArrayAsyncExtensions.ReadFromFileAsync("large-dataset.bin", cts.Token);
    ProcessLargeDataset(largeFile);
}
catch (OperationCanceledException)
{
    Console.WriteLine("File read timed out after 5 minutes");
}

// Parallel file reading
string[] filePaths = GetFilePaths();
var readTasks = filePaths.Select(async path => new {
    Path = path,
    Data = await ByteArrayAsyncExtensions.ReadFromFileAsync(path)
});

var results = await Task.WhenAll(readTasks);
foreach (var result in results)
{
    Console.WriteLine($"{result.Path}: {result.Data.Length} bytes");
}

// Configuration loading with fallback
byte[] configData;
try
{
    configData = await ByteArrayAsyncExtensions.ReadFromFileAsync("app.config");
}
catch (FileNotFoundException)
{
    Console.WriteLine("Config file not found, using defaults");
    configData = GetDefaultConfiguration();
}

var config = ParseConfiguration(configData);

// Image processing pipeline
string[] imageFiles = Directory.GetFiles("images", "*.jpg");
var processedImages = new List<byte[]>();

foreach (string imagePath in imageFiles)
{
    byte[] imageData = await ByteArrayAsyncExtensions.ReadFromFileAsync(imagePath);
    byte[] processed = await ProcessImageAsync(imageData);
    processedImages.Add(processed);

    string outputPath = Path.ChangeExtension(imagePath, ".processed.jpg");
    await processed.WriteToFileAsync(outputPath);
}

Async Stream Operations

WriteToStreamAsync

WriteToStreamAsync

Asynchronously writes byte array data to any stream.

Usage Examples

// Network stream writing
byte[] messageData = GetProtocolMessage();
using var tcpClient = new TcpClient("server.example.com", 8080);
using var networkStream = tcpClient.GetStream();

await messageData.WriteToStreamAsync(networkStream);
Console.WriteLine("Message sent to server");

// HTTP response streaming
public async Task StreamDataToResponse(HttpResponse response, byte[] data)
{
    response.ContentType = "application/octet-stream";
    response.ContentLength = data.Length;

    await data.WriteToStreamAsync(response.Body);
}

// Memory stream for in-memory operations
byte[] sourceData = GetSourceData();
using var memoryStream = new MemoryStream();

await sourceData.WriteToStreamAsync(memoryStream);
memoryStream.Position = 0;

// Process the stream data
byte[] processedData = await ProcessStreamAsync(memoryStream);

// File stream with custom buffer
byte[] largeData = GetLargeDataset();
using var fileStream = new FileStream("output.dat", FileMode.Create, FileAccess.Write, FileShare.None, bufferSize: 65536);

await largeData.WriteToStreamAsync(fileStream);
Console.WriteLine("Large dataset written to file");

ReadFromStreamAsync

ReadFromStreamAsync

Asynchronously reads all data from a stream into a byte array.

Usage Examples

// Network stream reading
using var tcpClient = new TcpClient("api.example.com", 443);
using var networkStream = tcpClient.GetStream();

byte[] responseData = await ByteArrayAsyncExtensions.ReadFromStreamAsync(networkStream);
string response = Encoding.UTF8.GetString(responseData);
Console.WriteLine($"Received: {response}");

// HTTP request body reading
public async Task<IActionResult> ProcessUpload()
{
    byte[] requestBody = await ByteArrayAsyncExtensions.ReadFromStreamAsync(Request.Body);

    // Process the uploaded data
    var result = ProcessBinaryData(requestBody);
    return Ok(result);
}

// Custom buffer size for memory efficiency
using var largeFileStream = new FileStream("large-file.bin", FileMode.Open);
byte[] fileData = await ByteArrayAsyncExtensions.ReadFromStreamAsync(
    largeFileStream,
    bufferSize: 1024 * 1024); // 1MB buffer

Console.WriteLine($"Read {fileData.Length} bytes with 1MB buffer");

// Cancellation support
using var cts = new CancellationTokenSource(TimeSpan.FromMinutes(2));
try
{
    byte[] streamData = await ByteArrayAsyncExtensions.ReadFromStreamAsync(
        inputStream,
        bufferSize: 8192,
        cts.Token);
}
catch (OperationCanceledException)
{
    Console.WriteLine("Stream read operation cancelled");
}

CopyToStreamAsync

CopyToStreamAsync

Asynchronously copies byte array data to a stream in configurable chunks.

Usage Examples

// Large file transfer with progress
byte[] largeFile = await ByteArrayAsyncExtensions.ReadFromFileAsync("source.bin");
using var destinationStream = new FileStream("destination.bin", FileMode.Create);

// Copy in 64KB chunks for better memory usage
await largeFile.CopyToStreamAsync(destinationStream, chunkSize: 65536);
Console.WriteLine("Large file copied successfully");

// Network transfer with bandwidth control
public async Task TransferWithThrottling(byte[] data, Stream destination)
{
    const int chunkSize = 4096; // 4KB chunks
    var delay = TimeSpan.FromMilliseconds(10); // Throttle bandwidth

    for (int offset = 0; offset < data.Length; offset += chunkSize)
    {
        int length = Math.Min(chunkSize, data.Length - offset);
        var chunk = data.AsSpan(offset, length).ToArray();

        await chunk.WriteToStreamAsync(destination);
        await Task.Delay(delay); // Bandwidth throttling
    }
}

// Progress reporting during copy
public async Task CopyWithProgress(byte[] data, Stream destination, IProgress<int> progress)
{
    const int chunkSize = 8192;
    int totalBytes = data.Length;
    int bytesCopied = 0;

    for (int offset = 0; offset < data.Length; offset += chunkSize)
    {
        int length = Math.Min(chunkSize, data.Length - offset);
        var chunk = data.AsSpan(offset, length).ToArray();

        await chunk.WriteToStreamAsync(destination);

        bytesCopied += length;
        int percentage = (int)((double)bytesCopied / totalBytes * 100);
        progress?.Report(percentage);
    }
}

Async Parallel Processing

ProcessInParallelAsync

ProcessInParallelAsync

Processes byte array data in parallel chunks with concurrency control.

Usage Examples

// Image processing in parallel
byte[] imageData = await ByteArrayAsyncExtensions.ReadFromFileAsync("large-image.raw");

// Process image tiles in parallel
var processedTiles = await imageData.ProcessInParallelAsync(
    async chunk => await ProcessImageTileAsync(chunk),
    chunkSize: 1024 * 1024, // 1MB tiles
    maxConcurrency: Environment.ProcessorCount);

Console.WriteLine($"Processed {processedTiles.Length} image tiles");

// Data analysis with parallel processing
byte[] sensorData = GetSensorReadings();

var analysisResults = await sensorData.ProcessInParallelAsync(
    async dataChunk => {
        // CPU-intensive analysis on each chunk
        return await AnalyzeSensorDataAsync(dataChunk);
    },
    chunkSize: 4096,
    maxConcurrency: 4);

// Aggregate results
var overallAnalysis = AggregateAnalysisResults(analysisResults);
Console.WriteLine($"Analysis complete: {overallAnalysis}");

// Cryptographic operations in parallel
byte[] largeDataset = GetLargeDataset();

var hashResults = await largeDataset.ProcessInParallelAsync(
    async chunk => await ComputeChunkHashAsync(chunk),
    chunkSize: 64 * 1024, // 64KB chunks
    maxConcurrency: 8);

// Combine chunk hashes
byte[] finalHash = CombineHashes(hashResults);

// Progress tracking with parallel processing
int completedChunks = 0;
var progress = new Progress<int>(count => {
    Interlocked.Increment(ref completedChunks);
    Console.WriteLine($"Completed {completedChunks} chunks");
});

var results = await data.ProcessInParallelAsync(
    async chunk => {
        var result = await ProcessChunkAsync(chunk);
        progress.Report(1);
        return result;
    },
    chunkSize: 8192,
    maxConcurrency: Environment.ProcessorCount);

TransformAsync

TransformAsync

Asynchronously transforms byte array data using a transformation function.

Usage Examples

// Data encryption transformation
byte[] plainData = GetSensitiveData();

byte[] encryptedData = await plainData.TransformAsync(
    async chunk => await EncryptChunkAsync(chunk, encryptionKey),
    chunkSize: 16384); // 16KB chunks

Console.WriteLine($"Encrypted {plainData.Length} bytes -> {encryptedData.Length} bytes");

// Compression transformation
byte[] uncompressedData = await ByteArrayAsyncExtensions.ReadFromFileAsync("large-file.txt");

byte[] compressedData = await uncompressedData.TransformAsync(
    async chunk => await CompressChunkAsync(chunk),
    chunkSize: 32768); // 32KB chunks

Console.WriteLine($"Compression ratio: {(double)compressedData.Length / uncompressedData.Length:P2}");

// Image format conversion
byte[] rawImageData = GetRawImageData();

byte[] jpegData = await rawImageData.TransformAsync(
    async chunk => await ConvertToJpegChunkAsync(chunk),
    chunkSize: 1024 * 1024); // 1MB chunks

// Data validation and cleaning
byte[] rawSensorData = GetRawSensorData();

byte[] cleanedData = await rawSensorData.TransformAsync(
    async chunk => {
        // Validate and clean each chunk
        var validatedChunk = ValidateDataChunk(chunk);
        var cleanedChunk = await CleanDataAsync(validatedChunk);
        return NormalizeDataChunk(cleanedChunk);
    },
    chunkSize: 8192);

// Protocol message transformation
byte[] rawMessages = GetRawMessages();

byte[] protocolMessages = await rawMessages.TransformAsync(
    async chunk => {
        var parsedMessages = ParseMessageChunk(chunk);
        var processedMessages = await ProcessMessagesAsync(parsedMessages);
        return SerializeMessages(processedMessages);
    },
    chunkSize: 4096);

Async Cryptographic Operations

ComputeHashAsync

ComputeHashAsync

Asynchronously computes hash values using various algorithms.

Usage Examples

// File integrity checking
byte[] fileData = await ByteArrayAsyncExtensions.ReadFromFileAsync("important.zip");

// Compute different hash types
byte[] sha256Hash = await fileData.ComputeHashAsync("SHA256");
byte[] sha1Hash = await fileData.ComputeHashAsync("SHA1");
byte[] md5Hash = await fileData.ComputeHashAsync("MD5");

Console.WriteLine($"SHA-256: {Convert.ToHexString(sha256Hash)}");
Console.WriteLine($"SHA-1: {Convert.ToHexString(sha1Hash)}");
Console.WriteLine($"MD5: {Convert.ToHexString(md5Hash)}");

// Batch hash computation
string[] files = Directory.GetFiles("documents", "*.pdf");
var hashTasks = files.Select(async file => {
    byte[] data = await ByteArrayAsyncExtensions.ReadFromFileAsync(file);
    byte[] hash = await data.ComputeHashAsync("SHA256");
    return new { File = file, Hash = Convert.ToHexString(hash) };
});

var hashResults = await Task.WhenAll(hashTasks);
foreach (var result in hashResults)
{
    Console.WriteLine($"{result.File}: {result.Hash}");
}

// Password hashing for authentication
public async Task<string> HashPasswordAsync(string password)
{
    byte[] passwordBytes = Encoding.UTF8.GetBytes(password);
    byte[] salt = GenerateRandomSalt();

    // Combine password and salt
    byte[] combined = new byte[passwordBytes.Length + salt.Length];
    Array.Copy(passwordBytes, 0, combined, 0, passwordBytes.Length);
    Array.Copy(salt, 0, combined, passwordBytes.Length, salt.Length);

    byte[] hash = await combined.ComputeHashAsync("SHA256");

    // Return salt + hash for storage
    return Convert.ToBase64String(salt) + ":" + Convert.ToBase64String(hash);
}

// Digital signature verification
public async Task<bool> VerifyDigitalSignature(byte[] data, byte[] signature)
{
    byte[] dataHash = await data.ComputeHashAsync("SHA256");

    // Verify signature against hash (simplified)
    return await VerifySignatureAsync(dataHash, signature);
}

VerifyIntegrityAsync

VerifyIntegrityAsync

Asynchronously verifies data integrity against expected hash values.

Usage Examples

// File download verification
public async Task<bool> VerifyDownloadedFile(string filePath, string expectedSha256)
{
    byte[] fileData = await ByteArrayAsyncExtensions.ReadFromFileAsync(filePath);
    byte[] expectedHash = Convert.FromHexString(expectedSha256);

    bool isValid = await fileData.VerifyIntegrityAsync(expectedHash, "SHA256");

    if (!isValid)
    {
        Console.WriteLine($"File {filePath} failed integrity check!");
        File.Delete(filePath); // Remove corrupted file
    }

    return isValid;
}

// Backup verification
public async Task VerifyBackupIntegrity()
{
    var backupManifest = await LoadBackupManifest();

    foreach (var entry in backupManifest.Files)
    {
        byte[] fileData = await ByteArrayAsyncExtensions.ReadFromFileAsync(entry.Path);
        byte[] expectedHash = Convert.FromHexString(entry.Sha256Hash);

        bool isValid = await fileData.VerifyIntegrityAsync(expectedHash, "SHA256");

        if (isValid)
        {
            Console.WriteLine($"✓ {entry.Path}");
        }
        else
        {
            Console.WriteLine($"✗ {entry.Path} - CORRUPTED!");
        }
    }
}

// Network message verification
public async Task<bool> VerifyNetworkMessage(byte[] messageData, byte[] providedHash)
{
    return await messageData.VerifyIntegrityAsync(providedHash, "SHA256");
}

// Database record integrity
public async Task<bool> VerifyDatabaseRecord(DatabaseRecord record)
{
    byte[] recordData = SerializeRecord(record);
    byte[] storedHash = Convert.FromBase64String(record.IntegrityHash);

    return await recordData.VerifyIntegrityAsync(storedHash, "SHA256");
}

Async Utility Operations

IndexOfAsync

IndexOfAsync

Asynchronously searches for patterns in byte arrays.

Usage Examples

// Log file pattern searching
byte[] logFile = await ByteArrayAsyncExtensions.ReadFromFileAsync("application.log");
byte[] errorPattern = Encoding.UTF8.GetBytes("ERROR");

int errorIndex = await logFile.IndexOfAsync(errorPattern);
if (errorIndex >= 0)
{
    Console.WriteLine($"First error found at position {errorIndex}");
}

// Binary protocol message parsing
byte[] networkData = await ReadNetworkDataAsync();
byte[] messageHeader = { 0xAA, 0xBB, 0xCC, 0xDD };

int headerIndex = await networkData.IndexOfAsync(messageHeader);
if (headerIndex >= 0)
{
    Console.WriteLine($"Message header found at position {headerIndex}");
    byte[] messagePayload = networkData.Skip(headerIndex + 4).ToArray();
}

// File format signature detection
byte[] fileData = await ByteArrayAsyncExtensions.ReadFromFileAsync("unknown.file");

// Check for various file signatures
var signatures = new Dictionary<string, byte[]>
{
    { "PNG", new byte[] { 0x89, 0x50, 0x4E, 0x47 } },
    { "JPEG", new byte[] { 0xFF, 0xD8, 0xFF } },
    { "PDF", Encoding.ASCII.GetBytes("%PDF") },
    { "ZIP", new byte[] { 0x50, 0x4B, 0x03, 0x04 } }
};

foreach (var signature in signatures)
{
    int index = await fileData.IndexOfAsync(signature.Value);
    if (index == 0) // Found at beginning
    {
        Console.WriteLine($"File type detected: {signature.Key}");
        break;
    }
}

SequenceEqualAsync

SequenceEqualAsync

Asynchronously compares byte arrays for equality.

Usage Examples

// File comparison
byte[] file1 = await ByteArrayAsyncExtensions.ReadFromFileAsync("version1.dat");
byte[] file2 = await ByteArrayAsyncExtensions.ReadFromFileAsync("version2.dat");

bool areEqual = await file1.SequenceEqualAsync(file2);
if (areEqual)
{
    Console.WriteLine("Files are identical");
}
else
{
    Console.WriteLine("Files differ");
}

// Backup verification
public async Task<bool> VerifyBackup(string originalPath, string backupPath)
{
    byte[] original = await ByteArrayAsyncExtensions.ReadFromFileAsync(originalPath);
    byte[] backup = await ByteArrayAsyncExtensions.ReadFromFileAsync(backupPath);

    return await original.SequenceEqualAsync(backup);
}

// Network message validation
public async Task<bool> ValidateRetransmission(byte[] originalMessage, byte[] retransmittedMessage)
{
    return await originalMessage.SequenceEqualAsync(retransmittedMessage);
}

// Configuration change detection
public async Task<bool> HasConfigurationChanged()
{
    byte[] currentConfig = await ByteArrayAsyncExtensions.ReadFromFileAsync("current.config");
    byte[] lastKnownConfig = await ByteArrayAsyncExtensions.ReadFromFileAsync("last-known.config");

    bool hasChanged = !(await currentConfig.SequenceEqualAsync(lastKnownConfig));

    if (hasChanged)
    {
        await currentConfig.WriteToFileAsync("last-known.config"); // Update baseline
    }

    return hasChanged;
}

Advanced Usage Patterns

High-Performance File Processing

public async Task ProcessLargeFiles(string[] filePaths)
{
    // Process multiple large files concurrently
    var semaphore = new SemaphoreSlim(Environment.ProcessorCount, Environment.ProcessorCount);

    var tasks = filePaths.Select(async filePath => {
        await semaphore.WaitAsync();
        try
        {
            // Read file asynchronously
            byte[] data = await ByteArrayAsyncExtensions.ReadFromFileAsync(filePath);

            // Process in parallel chunks
            var results = await data.ProcessInParallelAsync(
                async chunk => await ProcessDataChunkAsync(chunk),
                chunkSize: 1024 * 1024, // 1MB chunks
                maxConcurrency: 4);

            // Transform results
            byte[] processedData = await data.TransformAsync(
                async chunk => await ApplyTransformationAsync(chunk),
                chunkSize: 512 * 1024);

            // Write processed data
            string outputPath = Path.ChangeExtension(filePath, ".processed");
            await processedData.WriteToFileAsync(outputPath);

            Console.WriteLine($"Processed: {filePath}");
        }
        finally
        {
            semaphore.Release();
        }
    });

    await Task.WhenAll(tasks);
}

Distributed Data Processing

public async Task ProcessDistributedDataset(byte[] dataset)
{
    // Split dataset for distributed processing
    const int nodeCount = 4;
    const int chunkSize = dataset.Length / nodeCount;

    var processingTasks = new List<Task<byte[]>>();

    for (int i = 0; i < nodeCount; i++)
    {
        int startIndex = i * chunkSize;
        int length = (i == nodeCount - 1) ? dataset.Length - startIndex : chunkSize;

        byte[] chunk = new byte[length];
        Array.Copy(dataset, startIndex, chunk, 0, length);

        // Process each chunk on a different "node" (async operation)
        processingTasks.Add(ProcessOnNodeAsync(chunk, nodeId: i));
    }

    // Wait for all nodes to complete
    byte[][] results = await Task.WhenAll(processingTasks);

    // Combine results
    byte[] finalResult = ByteArrayExtensions.Concatenate(results);

    Console.WriteLine($"Distributed processing complete: {finalResult.Length} bytes");
}

async Task<byte[]> ProcessOnNodeAsync(byte[] data, int nodeId)
{
    Console.WriteLine($"Node {nodeId} processing {data.Length} bytes");

    // Simulate distributed processing
    await Task.Delay(TimeSpan.FromSeconds(1 + nodeId)); // Different processing times

    // Transform data asynchronously
    return await data.TransformAsync(
        async chunk => await SimulateProcessingAsync(chunk),
        chunkSize: 4096);
}

Performance Characteristics

Operation	Complexity	Notes
WriteToFileAsync	O(n)	I/O bound, benefits from async
ReadFromFileAsync	O(n)	I/O bound, non-blocking
ProcessInParallelAsync	O(n/p)	p = parallelism level
TransformAsync	O(n)	Sequential chunk processing
ComputeHashAsync	O(n)	CPU intensive, benefits from Task.Run
IndexOfAsync	O(n*m)	n = data length, m = pattern length

Best Practices

• Use cancellation tokens for all operations to enable timeout and cancellation
• Configure concurrency limits based on system resources and workload
• Choose appropriate chunk sizes for memory efficiency vs. overhead
• Handle exceptions properly with try-catch blocks and proper cleanup
• Use progress reporting for long-running operations
• Prefer async methods in async contexts to avoid blocking
• Configure buffer sizes based on data size and memory constraints
• Monitor resource usage when processing large datasets

Common Use Cases

1. Web Applications: Async file uploads and downloads
2. Data Processing: Large dataset analysis and transformation
3. Backup Systems: File integrity verification and copying
4. Network Services: Protocol message processing
5. Content Management: Media file processing and conversion
6. Security Applications: Cryptographic operations and validation
7. IoT Systems: Sensor data processing and analysis
8. Scientific Computing: Parallel data analysis workflows

Error Handling Guidelines

• Wrap I/O operations in try-catch blocks for proper error handling
• Use specific exception types for different error scenarios
• Implement retry logic for transient failures
• Log errors appropriately with sufficient context
• Clean up resources in finally blocks or using statements
• Validate inputs before processing to prevent errors
• Handle cancellation gracefully with proper cleanup

See Also

• ByteArrayExtensions - For synchronous byte array operations
• ByteArrayBuilder - For constructing byte arrays
• ByteArrayCompressionExtensions - For async compression operations
• ByteArrayUtilities - For analysis and measurement tools