Optimize StackJob Sizes

[james7132]

Looking into async-task, I found it stores it's future and output as a union, while Forte stores the op to call and the output as a struct. This can help save stack space with deeper recursion trees.

Alternatively, if we know we're copying the return value somewhere, we can opt to directly write the output of the job to that location instead of to a space inside the job, writing to a *mut MaybeUninit<T> instead of a result value in the job.

[NthTensor]

Looking at join, I guess we allocate space for F once and Result<T, Box<dyn Any>> twice (one outside the job and one within it). The way to eliminate all of this duplication would be to follow both your suggestions: hold a pointer to a union over F and Result<T, Box<dyn Any>>.

union StackJobData<F, T> {
    func: F,
    result: Result<T, Box<dyn Any>>
}

impl<T, F> StackJobData<F, T> {
    fn new(func: F) -> StackJobData<F, T> {
        StackJobData { func }
    }
}

struct StackJob<F, T> {
    completed: Latch,
    data: *mut StackJobData<F, T>
}

impl<F, T> StackJob<F, T>
where
    F: FnOnce(&Worker) -> T + Send,
    T: Send,
{
    /// # Safety
    /// 
    /// The provided data must contain a function, not a result.
    unsafe fn new(data: &mut StackJobData, worker: &Worker) -> StackJob<F, T> {
        StackJob {
            data: ptr::from_mut(data),
            completed: worker.new_latch(),
        }
    }
}

Then again, I'm not sure how much LLVM is able to crunch this.

We can't have the job directly be a union because the latch can't be invalidated.

I have been having issues with stack overflows since switching to a "fat" stack job implementation, so I'll prioritize this.

[james7132]

I'd actually go one step further and lift the Result split and make the latch/signal a pseudo-enum discriminator:

union StackJobData<F, T> {
    func: F,
    output: T,
    error: Box<dyn Any>
}

That way you can directly return the output value without extra unwrapping.

[NthTensor]

I am getting reports of stack overflows when using very deep joins, especially on windows when in debug mode.

I'm going to move forward with this optimization, but I think we may also want to consider pushing large types into a heap-allocated pool (like https://docs.rs/infinity_pool/latest/infinity_pool/ perhaps).