Welcome Guest Search | Active Topics | Sign In | Register

EOWP Fallback Options
Stefan Koell
Posted: Tuesday, September 25, 2018 6:50:50 AM
Rank: Advanced Member
Groups: Member

Joined: 12/23/2013
Posts: 114
Hi,

because of user reports about memory issues with rundll for the chromium worker process, I switched to EOWP:
https://www.essentialobjects.com/doc/Common/eowp.html

This seesm to have fixed most of the issues but some customers are reporting issues when this process is blocked for some reasons (anti virus or black/white listing tools). If the process cannot be created all sorts of weird behavior (hang, crashes, etc.) can be observed. My question: would it be possible to handle this more gracefully and fall back to rundll in case the specified custom worker process cannot be started? Maybe notifying the webview through an event that this fallback happened?

Thanks,
Stefan
eo_support
Posted: Tuesday, September 25, 2018 9:31:02 AM
Rank: Administration
Groups: Administration

Joined: 5/27/2007
Posts: 24,221
Hi,

No. EOWP infact is the fallback for rundll. AntiVirus tends to flag rundll more frequently because its a system file and it is can be abused by malware. So if rundll fails, we will fall back to EOWP. So we can't fallback the other way.

Thanks!
Stefan Koell
Posted: Tuesday, September 25, 2018 9:38:22 AM
Rank: Advanced Member
Groups: Member

Joined: 12/23/2013
Posts: 114
Thanks for the quick reply!

When I set
EO.Base.Runtime.EnableEOWP = true;
shouldn't be the behavior reversed?

I mean, in this case, the configured EOWP should be preferred but if that fails, it should fall back to rundll?

Am I missing something? I've set the EnableEOWP to true and called InitWorkerProcessExecutable method. Still I have customers (who need explicit whitelisting of .exe files) where the complete app is crashing because the EOWP wasn't able to start. How can I make sure my app runs as well in those environments?

Regards,
Stefan
eo_support
Posted: Tuesday, September 25, 2018 8:20:06 PM
Rank: Administration
Groups: Administration

Joined: 5/27/2007
Posts: 24,221
The key point here is if they even raises a false alarm on a digitally signed and explicitly whitelisted eowp.exe, then there is no point to fall back to a generic purposed rundll32.exe since that file is much more likely to raises false alarm than eowp.exe. In this case you just have to find out who killed eowp.exe and stop that. We have no way of stopping someone killing our process. You will have to find out who did it and disable it.
Stefan Koell
Posted: Wednesday, September 26, 2018 5:34:32 AM
Rank: Advanced Member
Groups: Member

Joined: 12/23/2013
Posts: 114
On this particular system there are Software Restriction Policies active. The behavior is like this when I run it manually:
C:\path_to_wp> EOWP.exe
The output is:
This program is blocked by group policy. For more information, contact your system administrator.

So, as I see it, your component thinks the process has been started but it hasn't. I guess you can't detect if the process has actually been started or not but later on, your web view tries to communicate with the process and because the process is not there, the app will freeze and hang. It would be nice to have this situation handled gracefully by logging an error using the event and closing the web view or something. Hanging the app in this scenario is just bad behavior. I hope this makes sense.
eo_support
Posted: Wednesday, September 26, 2018 9:18:01 AM
Rank: Administration
Groups: Administration

Joined: 5/27/2007
Posts: 24,221
Thanks for your feedback. My understanding is we do have such code in place but we will double check.
Stefan Koell
Posted: Wednesday, September 26, 2018 9:26:29 AM
Rank: Advanced Member
Groups: Member

Joined: 12/23/2013
Posts: 114
I also checked again and it seems when I use the threadrunner, I can actually catch an exception when I create the web view but if I just use the web view UI control, I see some strange behavior resulting in an exception (with no stacktrace in my code) and my app behaves weird (freezes/hangs).
eo_support
Posted: Wednesday, September 26, 2018 9:38:27 AM
Rank: Administration
Groups: Administration

Joined: 5/27/2007
Posts: 24,221
You should be able to use EO.Base.Runtime.Exception event to catch such exceptions.

The fact that your application freezes/hangs are normal ---- since an exception has already occurred so nothing afterwards will work correctly. It could freeze, it could crash, or it may have other unpredictable behavior. There is no point trying to "correct" these behaviors after a fatal exception.
Stefan Koell
Posted: Wednesday, September 26, 2018 9:41:42 AM
Rank: Advanced Member
Groups: Member

Joined: 12/23/2013
Posts: 114
I will double check the event handler. I was hoping that the web view could be destroyed or something or at least display a message that the view crashed instead of freezing/crashing the app. I agree, if the web view cannot be created because of the worker process not available, it's a fatal error but crashing the whole app is kind of brutal. There is a lot of other stuff in the app, the user still may need to work with...
Stefan Koell
Posted: Thursday, September 27, 2018 5:24:11 AM
Rank: Advanced Member
Groups: Member

Joined: 12/23/2013
Posts: 114
I did some more tests and found out that in my test's I'm not able to reverse the setting EnableEOWP. I do something like this:

EO.Base.Runtime.EnableEOWP = true;
EO.Base.Runtime.InitWorkerProcessExecutable("somepath"));

new Thread(() =>
{
try
{
var threadRunner = new ThreadRunner();
var webView = threadRunner.CreateWebView();
webView.Destroy();
webView.Dispose();
threadRunner.Dispose();
}
catch (Exception ex)
{
EO.Base.Runtime.EnableEOWP = false;
}
}).Start();

Even when I set EnableEOWP to false afterwards, it still seems to go with the custom EOWP. I expected it to use rundll.exe again.

In case you are wondering: I do the check in a background thread during application startup because it's taking quite a long time to do this and I didn't want to affect this my app startup performance.
Guest
Posted: Thursday, September 27, 2018 6:12:38 AM
Rank: Guest
Groups: Guest

Joined: 5/27/2007
Posts: -34
eo_support wrote:
The fact that your application freezes/hangs are normal ---- since an exception has already occurred so nothing afterwards will work correctly. It could freeze, it could crash, or it may have other unpredictable behavior. There is no point trying to "correct" these behaviors after a fatal exception.


Simply, no.

EO is failing to check for errors in an operation that is an obvious candidate for errors, and one where such errors should be expected and handled accordingly so that the caller or user is able to take meaningful action. The error in question (ultimately a failure in CreateProcess/Ex) has no good reason for causing program state corruption.

You can test this error yourself very simply, by renaming/moving the worker EXE. Anything that causes CreateProcess/Ex to fail (SRP, AV, corrupt .exe …) will cause EO to break in a way that it very much should not. This is most definitely not a fatal error — this is a very basic, non-fatal error that can be handled very cleanly by EO and then the caller in turn. EO won't work in that configuration, but there is no excuse for destroying the process state. The freezes are so bad that (in the case of Stefan's application) the error report window freezes up. The user can't even send an error report!

I was able to guess that it was SRP causing the fault; the exception message didn't say what image it was trying to execute, simply that running some unnamed process failed for some reason (1260 or something — I am not a walking error dictionary). The exception should state all these details, along the lines of "Cannot start EssentialObjects worker process at $PATH: $REASON (error code $CODE)". This way, the user can see what the program was trying to do, how and why, and what failed. (Or the caller can detect and handle this more cleanly if needed, as will be the case here.)

Error messages should not be guesswork.
eo_support
Posted: Thursday, September 27, 2018 7:46:29 AM
Rank: Administration
Groups: Administration

Joined: 5/27/2007
Posts: 24,221
You need to forget about falling back to rundll. You are really chasing something that does not exist. A fatal exception like this WILL put your application into an unpredictable and unrecoverable state. Just like you won't be able to save an air plane after an in air collision, you won't be able to recover your application after this exception either. This is one type of fatal exception ---- you can run into other situations like memory issues, permission issue, handle issues and such all of which can be fatal and unrecoverable and obviously its not possible for you to have fallback mechanism for all these. Because of this, setting EnableEOWP is one way and you can't reset it because even if you could reset it, many internal states have already been corrupted and won't work.

Whether using EOWP and rundll does not have much of direct impact on performance.
eo_support
Posted: Thursday, September 27, 2018 7:55:17 AM
Rank: Administration
Groups: Administration

Joined: 5/27/2007
Posts: 24,221
Daniel Beardsmore wrote:
eo_support wrote:
The fact that your application freezes/hangs are normal ---- since an exception has already occurred so nothing afterwards will work correctly. It could freeze, it could crash, or it may have other unpredictable behavior. There is no point trying to "correct" these behaviors after a fatal exception.


Simply, no.

EO is failing to check for errors in an operation that is an obvious candidate for errors, and one where such errors should be expected and handled accordingly so that the caller or user is able to take meaningful action. The error in question (ultimately a failure in CreateProcess/Ex) has no good reason for causing program state corruption.

You can test this error yourself very simply, by renaming/moving the worker EXE. Anything that causes CreateProcess/Ex to fail (SRP, AV, corrupt .exe …) will cause EO to break in a way that it very much should not. This is most definitely not a fatal error — this is a very basic, non-fatal error that can be handled very cleanly by EO and then the caller in turn. EO won't work in that configuration, but there is no excuse for destroying the process state. The freezes are so bad that (in the case of Stefan's application) the error report window freezes up. The user can't even send an error report!

I was able to guess that it was SRP causing the fault; the exception message didn't say what image it was trying to execute, simply that running some unnamed process failed for some reason (1260 or something — I am not a walking error dictionary). The exception should state all these details, along the lines of "Cannot start EssentialObjects worker process at $PATH: $REASON (error code $CODE)". This way, the user can see what the program was trying to do, how and why, and what failed. (Or the caller can detect and handle this more cleanly if needed, as will be the case here.)

Error messages should not be guesswork.


We are not aware that the error message box we displayed would freeze up. We will follow your steps and see if we can reproduce the problem. If we can not, then it may have something specific related to your application and we may need to create a test app for us.

Your suggestion about error message also makes sense and we will look into that as well.

We disagree with you that such an error is definitely not fatal --- for any application if you a core file is corrupted/missing/non-accessible then it is fatal. We have already spent much time explaining this to Stefan. So we won't be repeating that here again.
Stefan Koell
Posted: Thursday, September 27, 2018 8:12:12 AM
Rank: Advanced Member
Groups: Member

Joined: 12/23/2013
Posts: 114
Ok, let's take a deep breath and take a step back. At some point you introduced an option to use an external worker process by setting the EnableEOWP to true. Since I implemented to use the external worker process, I get these issues on some systems with very restrictive policies.

Coming back to my question I posted earlier today: is there a way to reverse the behavior once I found out that the external worker process is not allowed to run? I just want to get back to the previous behavior and set EnableEOWP to false and the component should respect that. If so, all is good and I can fall back to rundll.exe myself (which works in these restricted environments). That's all I'm asking for. If I can do that, there will be no hang and no further discussion if the component should handle this differently or not.
Guest
Posted: Thursday, September 27, 2018 8:13:31 AM
Rank: Guest
Groups: Guest

Joined: 5/27/2007
Posts: -34
eo_support wrote:
We disagree with you that such an error is definitely not fatal --- for any application if you a core file is corrupted/missing/non-accessible then it is fatal. We have already spent much time explaining this to Stefan. So we won't be repeating that here again.


I don't believe anyone ever asserted that this is a core file. A program can do many things at once, and offer many features that rely on many third-party components.

I must reiterate: failing to start this executable will not do the slightest bit of harm to the runtime state of a correctly-implemented program. The feature that depends on it will not function, but this will not by necessity destabilise the program unless it contains fundamental programming errors. Program freezes, crashes and erratic behaviour are the result of bad programming. A correctly-written program will not behave like this when an error occurs, especially a predictable error such as this one.

How the calling program chooses to handle an inoperative EO instance is entirely down to the calling program, and whether EO is a core component or not. The caller may choose to log and report an error and terminate, or it may choose to switch off the feature that cannot be used. That is the calling program's choice, and it is perfectly possible to make all these choices in a program without going off the rails and crashing.
eo_support
Posted: Thursday, September 27, 2018 9:03:20 AM
Rank: Administration
Groups: Administration

Joined: 5/27/2007
Posts: 24,221
Let's agree to disagree. We must maintain the product in a manageable manner and that means it's not possible for us to give every user what they want. In your case I can confirm that these two things will not change in the near future:

1. There will be no fallback from EnableEOWP being true and then being false;
2. Once eowp.exe fails, EO.WebBrowser component is not recoverable and the behavior is unpredictable;

We have already explained to you our reasons behind these decisions. You can certainly disagree with us but these decisions are final and will not change.

Please consider this issue closed. We will not comment on this issue further.


You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.