Welcome Guest Search | Active Topics | Sign In | Register

Out of memory exception Options
test_eo
Posted: Friday, August 5, 2022 2:45:08 AM
Rank: Newbie
Groups: Member

Joined: 8/5/2022
Posts: 1
Hi, how do I dispose or shutdown a webview coz it is not working due to out of memory exception (OOME).
here is my code which is working unless I have multiple scraper running. it triggers the out of memory error, and when I checked my task explorer, it has around 20 EO worker process taking up around 89% of memory usage.
I tried adding a
Quote:
webView.Close(true);
but it is giving me an error "Destroy must be called from the same thread the WebView is created". Can someone pls tell me how to dispose and possibly avoid OOME? thanks
Quote:
EO.WebEngine.Engine engine = EO.WebEngine.Engine.Create("enginetest");
using (threadRunner = new ThreadRunner("test thread", engine))
{
var proxy = ProxyHelper.GetProxy();

if (threadRunner.Engine != null)
{
threadRunner.Engine.Options.Proxy = new EO.Base.ProxyInfo(EO.Base.ProxyType.HTTP, proxy.Address,
proxy.Port, proxy.Username, proxy.Password);
}


using (WebView webView = threadRunner.CreateWebView(new EO.WebEngine.BrowserOptions() { LoadImages = false }))
{
var customResourceHander = new CustomResourceHandler();

webView.RegisterResourceHandler(customResourceHander);

threadRunner.Send(async () =>
{
await webView.LoadUrlAsync(url);
});

int waitCounter = 0;

while (customResourceHander.Content == string.Empty || waitCounter++ > 30)
{
Thread.Sleep(200);
}
threadRunner.Dispose();
return customResourceHander.Content;
}
}
eo_support
Posted: Friday, August 5, 2022 10:59:23 AM
Rank: Administration
Groups: Administration

Joined: 5/27/2007
Posts: 24,217
Hi,

Your code is correct. When you use using the WebView will be automatically destroyed by the ThreadRunner. So there is no need for you to call WebView.Close directly.

As long as you have multiple concurrent scrapper, you will have many worker processes. So the only way for you to avoid an out of memory error is probably to reduce the number of concurrent scrappers.

The other option you can try is not to create new Engine/WebView for every request. You can change the Engine's Proxy setting at runtime and it will take effect for the next load request.

Thanks!


You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.