Support async methods in usage scenarios of Throws constraints

Hi Simone,

As you have mentioned, the statement Assert is used on whether the async
lambda throws a specific exception or not but since the lambda itself is
async-ed, do you mean to say that the assertion itself should be awaited ?

Jv

On Thu, Oct 25, 2012 at 12:51 PM, Simone Busoli
<email address hidden>wrote:

> Public bug reported:
>
> Although we support async test methods, we're not handling other async
> things properly. For instance this is not working as expected:
>
> Assert.That(async () => await DoSomething(), Throws.Exception);
>
> The reasons is the same as why we didn't support async methods: we are
> not waiting until the function has completed its execution.
>
> Additional details here: http://stackoverflow.com/questions/13056716
> /async-unit-tests-not-working-as-expected
>
> ** Affects: nunitv2
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are a member of NUnit
> Developers, which is subscribed to NUnit V2.
> https://bugs.launchpad.net/bugs/1071164
>
> Title:
> Support async methods in usage scenarios of Throws constraints
>
> Status in NUnit V2 Test Framework:
> New
>
> Bug description:
> Although we support async test methods, we're not handling other async
> things properly. For instance this is not working as expected:
>
> Assert.That(async () => await DoSomething(), Throws.Exception);
>
> The reasons is the same as why we didn't support async methods: we are
> not waiting until the function has completed its execution.
>
> Additional details here: http://stackoverflow.com/questions/13056716
> /async-unit-tests-not-working-as-expected
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunitv2/+bug/1071164/+subscriptions
>

--
Regards,

Ravichandran Jv
http://ravichandranjv.blogspot.com

No. The lambda itself should be waited for completion.

On Thu, Oct 25, 2012 at 11:43 AM, Jv <email address hidden> wrote:

> Hi Simone,
>
> As you have mentioned, the statement Assert is used on whether the async
> lambda throws a specific exception or not but since the lambda itself is
> async-ed, do you mean to say that the assertion itself should be awaited ?
>
> Jv
>
> On Thu, Oct 25, 2012 at 12:51 PM, Simone Busoli
> <email address hidden>wrote:
>
> > Public bug reported:
> >
> > Although we support async test methods, we're not handling other async
> > things properly. For instance this is not working as expected:
> >
> > Assert.That(async () => await DoSomething(), Throws.Exception);
> >
> > The reasons is the same as why we didn't support async methods: we are
> > not waiting until the function has completed its execution.
> >
> > Additional details here: http://stackoverflow.com/questions/13056716
> > /async-unit-tests-not-working-as-expected
> >
> > ** Affects: nunitv2
> > Importance: Undecided
> > Status: New
> >
> > --
> > You received this bug notification because you are a member of NUnit
> > Developers, which is subscribed to NUnit V2.
> > https://bugs.launchpad.net/bugs/1071164
> >
> > Title:
> > Support async methods in usage scenarios of Throws constraints
> >
> > Status in NUnit V2 Test Framework:
> > New
> >
> > Bug description:
> > Although we support async test methods, we're not handling other async
> > things properly. For instance this is not working as expected:
> >
> > Assert.That(async () => await DoSomething(), Throws.Exception);
> >
> > The reasons is the same as why we didn't support async methods: we are
> > not waiting until the function has completed its execution.
> >
> > Additional details here: http://stackoverflow.com/questions/13056716
> > /async-unit-tests-not-working-as-expected
> >
> > To manage notifications about this bug go to:
> > https://bugs.launchpad.net/nunitv2/+bug/1071164/+subscriptions
> >
>
>
> --
> Regards,
>
> Ravichandran Jv
> http://ravichandranjv.blogspot.com
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1071164
>
> Title:
> Support async methods in usage scenarios of Throws constraints
>
> Status in NUnit V2 Test Framework:
> New
>
> Bug description:
> Although we support async test methods, we're not handling other async
> things properly. For instance this is not working as expected:
>
> Assert.That(async () => await DoSomething(), Throws.Exception);
>
> The reasons is the same as why we didn't support async methods: we are
> not waiting until the function has completed its execution.
>
> Additional details here: http://stackoverflow.com/questions/13056716
> /async-unit-tests-not-working-as-expected
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunitv2/+bug/1071164/+subscriptions
>

Yes, but since the async lambda is within the Assert statement, it means
that either the "Assert" should be awaited as "await Assert" or the [Test]
attribute should be with [await Test} because the "async" is part of the
CLR and not the NUnit framework.

If the lambda is to be awaited, it means that the CLR would await it, which
is how an unhandled exception works, no?

Jv

On Thu, Oct 25, 2012 at 4:54 PM, Simone Busoli
<email address hidden>wrote:

> No. The lambda itself should be waited for completion.
>
> On Thu, Oct 25, 2012 at 11:43 AM, Jv <email address hidden> wrote:
>
> > Hi Simone,
> >
> > As you have mentioned, the statement Assert is used on whether the async
> > lambda throws a specific exception or not but since the lambda itself is
> > async-ed, do you mean to say that the assertion itself should be awaited
> ?
> >
> > Jv
> >
> > On Thu, Oct 25, 2012 at 12:51 PM, Simone Busoli
> > <email address hidden>wrote:
> >
> > > Public bug reported:
> > >
> > > Although we support async test methods, we're not handling other async
> > > things properly. For instance this is not working as expected:
> > >
> > > Assert.That(async () => await DoSomething(), Throws.Exception);
> > >
> > > The reasons is the same as why we didn't support async methods: we are
> > > not waiting until the function has completed its execution.
> > >
> > > Additional details here: http://stackoverflow.com/questions/13056716
> > > /async-unit-tests-not-working-as-expected
> > >
> > > ** Affects: nunitv2
> > > Importance: Undecided
> > > Status: New
> > >
> > > --
> > > You received this bug notification because you are a member of NUnit
> > > Developers, which is subscribed to NUnit V2.
> > > https://bugs.launchpad.net/bugs/1071164
> > >
> > > Title:
> > > Support async methods in usage scenarios of Throws constraints
> > >
> > > Status in NUnit V2 Test Framework:
> > > New
> > >
> > > Bug description:
> > > Although we support async test methods, we're not handling other
> async
> > > things properly. For instance this is not working as expected:
> > >
> > > Assert.That(async () => await DoSomething(), Throws.Exception);
> > >
> > > The reasons is the same as why we didn't support async methods: we
> are
> > > not waiting until the function has completed its execution.
> > >
> > > Additional details here: http://stackoverflow.com/questions/13056716
> > > /async-unit-tests-not-working-as-expected
> > >
> > > To manage notifications about this bug go to:
> > > https://bugs.launchpad.net/nunitv2/+bug/1071164/+subscriptions
> > >
> >
> >
> > --
> > Regards,
> >
> > Ravichandran Jv
> > http://ravichandranjv.blogspot.com
> >
> > --
> > You received this bug notification because you are subscribed to the bug
> > report.
> > https://bugs.launchpad.net/bugs/1071164
> >
> > Title:
> > Support async methods in usage scenarios of Throws constraints
> >
> > Status in NUnit V2 Test Framework:
> > New
> >
> > Bug description:
> > Although we support async test methods, we're not handling other async
> > things properly. For instance this is not working as expected:
> >
> > Assert.That(async () => await DoSomething(), Th...

Jv, Assert.That is a static, non-async void method, there is no chance on
earth that you could await it. Also, to my best understanding attributes
cannot be awaited either.
If you have any basic questions about how async/await works you should ask
them on StackOverflow or similar websites, not here.

On Thu, Oct 25, 2012 at 1:38 PM, Jv <email address hidden> wrote:

> Yes, but since the async lambda is within the Assert statement, it means
> that either the "Assert" should be awaited as "await Assert" or the [Test]
> attribute should be with [await Test} because the "async" is part of the
> CLR and not the NUnit framework.
>
> If the lambda is to be awaited, it means that the CLR would await it, which
> is how an unhandled exception works, no?
>
> Jv
>
> On Thu, Oct 25, 2012 at 4:54 PM, Simone Busoli
> <email address hidden>wrote:
>
> > No. The lambda itself should be waited for completion.
> >
> > On Thu, Oct 25, 2012 at 11:43 AM, Jv <email address hidden> wrote:
> >
> > > Hi Simone,
> > >
> > > As you have mentioned, the statement Assert is used on whether the
> async
> > > lambda throws a specific exception or not but since the lambda itself
> is
> > > async-ed, do you mean to say that the assertion itself should be
> awaited
> > ?
> > >
> > > Jv
> > >
> > > On Thu, Oct 25, 2012 at 12:51 PM, Simone Busoli
> > > <email address hidden>wrote:
> > >
> > > > Public bug reported:
> > > >
> > > > Although we support async test methods, we're not handling other
> async
> > > > things properly. For instance this is not working as expected:
> > > >
> > > > Assert.That(async () => await DoSomething(), Throws.Exception);
> > > >
> > > > The reasons is the same as why we didn't support async methods: we
> are
> > > > not waiting until the function has completed its execution.
> > > >
> > > > Additional details here: http://stackoverflow.com/questions/13056716
> > > > /async-unit-tests-not-working-as-expected
> > > >
> > > > ** Affects: nunitv2
> > > > Importance: Undecided
> > > > Status: New
> > > >
> > > > --
> > > > You received this bug notification because you are a member of NUnit
> > > > Developers, which is subscribed to NUnit V2.
> > > > https://bugs.launchpad.net/bugs/1071164
> > > >
> > > > Title:
> > > > Support async methods in usage scenarios of Throws constraints
> > > >
> > > > Status in NUnit V2 Test Framework:
> > > > New
> > > >
> > > > Bug description:
> > > > Although we support async test methods, we're not handling other
> > async
> > > > things properly. For instance this is not working as expected:
> > > >
> > > > Assert.That(async () => await DoSomething(), Throws.Exception);
> > > >
> > > > The reasons is the same as why we didn't support async methods: we
> > are
> > > > not waiting until the function has completed its execution.
> > > >
> > > > Additional details here:
> http://stackoverflow.com/questions/13056716
> > > > /async-unit-tests-not-working-as-expected
> > > >
> > > > To manage notifications about this bug go to:
> > > > https://bugs.launchpad.net/nunitv2/+bug/1071164/+subscriptions
> > > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > Ravichandran Jv
> > > ...

Hi Simone,

What I am asking is related to the bug not to Assert.That or to how .net
works.

My question is not on how async/await works but on whether the bug that you
have reported is a bug at all. Because your response itself says it is
concerned with .net.

Jv

On Thu, Oct 25, 2012 at 5:42 PM, Simone Busoli
<email address hidden>wrote:

> Jv, Assert.That is a static, non-async void method, there is no chance on
> earth that you could await it. Also, to my best understanding attributes
> cannot be awaited either.
> If you have any basic questions about how async/await works you should ask
> them on StackOverflow or similar websites, not here.
>
> On Thu, Oct 25, 2012 at 1:38 PM, Jv <email address hidden> wrote:
>
> > Yes, but since the async lambda is within the Assert statement, it means
> > that either the "Assert" should be awaited as "await Assert" or the
> [Test]
> > attribute should be with [await Test} because the "async" is part of the
> > CLR and not the NUnit framework.
> >
> > If the lambda is to be awaited, it means that the CLR would await it,
> which
> > is how an unhandled exception works, no?
> >
> > Jv
> >
> > On Thu, Oct 25, 2012 at 4:54 PM, Simone Busoli
> > <email address hidden>wrote:
> >
> > > No. The lambda itself should be waited for completion.
> > >
> > > On Thu, Oct 25, 2012 at 11:43 AM, Jv <email address hidden>
> wrote:
> > >
> > > > Hi Simone,
> > > >
> > > > As you have mentioned, the statement Assert is used on whether the
> > async
> > > > lambda throws a specific exception or not but since the lambda itself
> > is
> > > > async-ed, do you mean to say that the assertion itself should be
> > awaited
> > > ?
> > > >
> > > > Jv
> > > >
> > > > On Thu, Oct 25, 2012 at 12:51 PM, Simone Busoli
> > > > <email address hidden>wrote:
> > > >
> > > > > Public bug reported:
> > > > >
> > > > > Although we support async test methods, we're not handling other
> > async
> > > > > things properly. For instance this is not working as expected:
> > > > >
> > > > > Assert.That(async () => await DoSomething(), Throws.Exception);
> > > > >
> > > > > The reasons is the same as why we didn't support async methods: we
> > are
> > > > > not waiting until the function has completed its execution.
> > > > >
> > > > > Additional details here:
> http://stackoverflow.com/questions/13056716
> > > > > /async-unit-tests-not-working-as-expected
> > > > >
> > > > > ** Affects: nunitv2
> > > > > Importance: Undecided
> > > > > Status: New
> > > > >
> > > > > --
> > > > > You received this bug notification because you are a member of
> NUnit
> > > > > Developers, which is subscribed to NUnit V2.
> > > > > https://bugs.launchpad.net/bugs/1071164
> > > > >
> > > > > Title:
> > > > > Support async methods in usage scenarios of Throws constraints
> > > > >
> > > > > Status in NUnit V2 Test Framework:
> > > > > New
> > > > >
> > > > > Bug description:
> > > > > Although we support async test methods, we're not handling other
> > > async
> > > > > things properly. For instance this is not working as expected:
> > > > >
> > > > > Assert.That(async () => await DoSomething(), Throws.Exception);
> > >...

Hi Charlie,

I'm working on this and I have one question. We basically need to do with Assert.Throws(async () => [some async code]) the same we're doing while executing tests, awaiting it. The code for the latter is in core, are there any existing ways by which we share code between the two? Since they obviously don't share any assembly references and it's only core which interacts with framework via reflection, the only thing which comes to mind is to just let the code leave in core and "reference" it from framework via "Add as link" feature in VS, which allows to use the same source file form both projects.

Thoughts?

Hi Simone,

Yes, that's what we already do with at least one source file, ITestCaseData.cs.
In NUnitLite and NUnit 3.0, the duplication can be eliminated.

Charlie

On Fri, Oct 26, 2012 at 1:49 PM, Simone Busoli
<email address hidden> wrote:
> Hi Charlie,
>
> I'm working on this and I have one question. We basically need to do
> with Assert.Throws(async () => [some async code]) the same we're doing
> while executing tests, awaiting it. The code for the latter is in core,
> are there any existing ways by which we share code between the two?
> Since they obviously don't share any assembly references and it's only
> core which interacts with framework via reflection, the only thing which
> comes to mind is to just let the code leave in core and "reference" it
> from framework via "Add as link" feature in VS, which allows to use the
> same source file form both projects.
>
> Thoughts?
>
> --
> You received this bug notification because you are subscribed to NUnit
> Extended Testing Platform.
> https://bugs.launchpad.net/bugs/1071164
>
> Title:
> Support async methods in usage scenarios of Throws constraints
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-3.0/+bug/1071164/+subscriptions

So I'm planning to do the same for this feature. The common code will
basically provide an operation to invoke a piece of code, and will wrap the
code to perform the asynchronous invocation, if needed.

Simone

On Fri, Oct 26, 2012 at 11:11 PM, Charlie Poole <email address hidden> wrote:

> Hi Simone,
>
> Yes, that's what we already do with at least one source file,
> ITestCaseData.cs.
> In NUnitLite and NUnit 3.0, the duplication can be eliminated.
>
> Charlie
>
> On Fri, Oct 26, 2012 at 1:49 PM, Simone Busoli
> <email address hidden> wrote:
> > Hi Charlie,
> >
> > I'm working on this and I have one question. We basically need to do
> > with Assert.Throws(async () => [some async code]) the same we're doing
> > while executing tests, awaiting it. The code for the latter is in core,
> > are there any existing ways by which we share code between the two?
> > Since they obviously don't share any assembly references and it's only
> > core which interacts with framework via reflection, the only thing which
> > comes to mind is to just let the code leave in core and "reference" it
> > from framework via "Add as link" feature in VS, which allows to use the
> > same source file form both projects.
> >
> > Thoughts?
> >
> > --
> > You received this bug notification because you are subscribed to NUnit
> > Extended Testing Platform.
> > https://bugs.launchpad.net/bugs/1071164
> >
> > Title:
> > Support async methods in usage scenarios of Throws constraints
> >
> > To manage notifications about this bug go to:
> > https://bugs.launchpad.net/nunit-3.0/+bug/1071164/+subscriptions
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1071164
>
> Title:
> Support async methods in usage scenarios of Throws constraints
>
> Status in NUnit Test Framework:
> Triaged
> Status in NUnitLite Testing Framework:
> Triaged
> Status in NUnit V2 Test Framework:
> In Progress
>
> Bug description:
> Although we support async test methods, we're not handling other async
> things properly. For instance this is not working as expected:
>
> Assert.That(async () => await DoSomething(), Throws.Exception);
>
> The reasons is the same as why we didn't support async methods: we are
> not waiting until the function has completed its execution.
>
> Additional details here: http://stackoverflow.com/questions/13056716
> /async-unit-tests-not-working-as-expected
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-3.0/+bug/1071164/+subscriptions
>

That sounds right to me.

Charlie

On Fri, Oct 26, 2012 at 2:41 PM, Simone Busoli
<email address hidden> wrote:
> So I'm planning to do the same for this feature. The common code will
> basically provide an operation to invoke a piece of code, and will wrap the
> code to perform the asynchronous invocation, if needed.
>
> Simone
>
> On Fri, Oct 26, 2012 at 11:11 PM, Charlie Poole <email address hidden>
> wrote:
>
>> Hi Simone,
>>
>> Yes, that's what we already do with at least one source file,
>> ITestCaseData.cs.
>> In NUnitLite and NUnit 3.0, the duplication can be eliminated.
>>
>> Charlie
>>
>> On Fri, Oct 26, 2012 at 1:49 PM, Simone Busoli
>> <email address hidden> wrote:
>> > Hi Charlie,
>> >
>> > I'm working on this and I have one question. We basically need to do
>> > with Assert.Throws(async () => [some async code]) the same we're doing
>> > while executing tests, awaiting it. The code for the latter is in core,
>> > are there any existing ways by which we share code between the two?
>> > Since they obviously don't share any assembly references and it's only
>> > core which interacts with framework via reflection, the only thing which
>> > comes to mind is to just let the code leave in core and "reference" it
>> > from framework via "Add as link" feature in VS, which allows to use the
>> > same source file form both projects.
>> >
>> > Thoughts?
>> >
>> > --
>> > You received this bug notification because you are subscribed to NUnit
>> > Extended Testing Platform.
>> > https://bugs.launchpad.net/bugs/1071164
>> >
>> > Title:
>> > Support async methods in usage scenarios of Throws constraints
>> >
>> > To manage notifications about this bug go to:
>> > https://bugs.launchpad.net/nunit-3.0/+bug/1071164/+subscriptions
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1071164
>>
>> Title:
>> Support async methods in usage scenarios of Throws constraints
>>
>> Status in NUnit Test Framework:
>> Triaged
>> Status in NUnitLite Testing Framework:
>> Triaged
>> Status in NUnit V2 Test Framework:
>> In Progress
>>
>> Bug description:
>> Although we support async test methods, we're not handling other async
>> things properly. For instance this is not working as expected:
>>
>> Assert.That(async () => await DoSomething(), Throws.Exception);
>>
>> The reasons is the same as why we didn't support async methods: we are
>> not waiting until the function has completed its execution.
>>
>> Additional details here: http://stackoverflow.com/questions/13056716
>> /async-unit-tests-not-working-as-expected
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/nunit-3.0/+bug/1071164/+subscriptions
>>
>
> --
> You received this bug notification because you are subscribed to NUnit
> Extended Testing Platform.
> https://bugs.launchpad.net/bugs/1071164
>
> Title:
> Support async methods in usage scenarios of Throws constraints
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-3.0/+bug/1071164/+subscriptions