Take screen-shot of webpage with Firefox and Geckodriver using Selenium with C#

Gecko is a web browser engine used in many applications developed by Mozilla Foundation and the Mozilla Corporation. It is a proxy for using W3C WebDriver-compatible clients to interact with Gecko-based browsers, in this case, Firefox. Gecko Driver acts as the bridge between your script in Selenium and the Firefox browser.

Firefox browser implements the WebDriver protocol using an executable called GeckoDriver.exe. This executable starts a server on your system through which all interaction happens. It translates calls into the Marionette automation protocol by acting as a proxy between the local and remote ends.

To generate the screenshot, the web page will be loaded in the Firefox WebDriver and the height of the window adjusted appropriately. Every GeckoDriver process spawns 4 Firefox processes which you can check through Task Manager. The Preview generation logic can be written as a Web Service and hosted on IIS. The AppPool would require to be run with LocalSystem permission to enable communication between GeckoDriver and Firefox WebDriver.

The below code will run Firefox WebDriver in headless mode:

public static IWebDriver driverWeb;
public static FirefoxOptions optionsFirefox;
static int totalWidth = 1024;
static int additionalHeight = 220;
static int additionalHeightOffset = 2180;
static int normalHeight = 768;
static int explicitWaitTime = 5;

Setup browser profile:

static Firefox()
{
	try
	{
		optionsFirefox = new FirefoxOptions();
		FirefoxProfile profile = new FirefoxProfile();
		profile.SetPreference("permissions.default.desktop-notification", 1);
		profile.AcceptUntrustedCertificates = true; //Accept SSL certificates which have expired
		profile.AssumeUntrustedCertificateIssuer = false; //Firefox assumes untrusted SSL certificates are coming from untrusted issuer 
		profile.SetPreference("general.useragent.override", Convert.ToString(ConfigurationManager.AppSettings["driverUserAgent"]));
		profile.SetPreference("layout.css.devPixelsPerPx", "0.9");
		optionsFirefox.Profile = profile;
		optionsFirefox.AddArgument("-headless");
		KillExistingProcesses();
		initializeWebDriver();
	}
	catch (Exception e)
	{
		throw e;
	}
}

Initialize Web Driver:

public static void initializeWebDriver()
{
	try
	{
		FirefoxDriverService service = FirefoxDriverService.CreateDefaultService(Convert.ToString(ConfigurationManager.AppSettings["driverPath"]));
		service.HideCommandPromptWindow = true;
		driverWeb = null;
		driverWeb = new FirefoxDriver(service, optionsFirefox, new TimeSpan(0, 0, Convert.ToInt16(ConfigurationManager.AppSettings["setDriverTimeout"])));
		driverWeb.Manage().Timeouts().ImplicitWait = new TimeSpan(0, 0, Convert.ToInt16(ConfigurationManager.AppSettings["setPageLoadTimeout"]));
		driverWeb.Manage().Timeouts().PageLoad = new TimeSpan(0, 0, Convert.ToInt16(ConfigurationManager.AppSettings["setPageLoadTimeout"]));
	}
	catch
	{
		//log exception.
	}
}

Generate Preview:

//This method will be called from a Rest Service and returns a byte array for image.
public byte[] getImage()
{
	byte[] imageDataWeb;
	try
	{
		if (driverWeb == null)
		{
			lock (padlockWeb)
			{
				//double lock check for thread-safety.
				if (driverWeb == null)
				{
					initializeWebDriver();
				}
			}
		}

		if (getHttpResponseCode(link))
		{
			lock (driverWeb)
			{
				driverWeb.Navigate().GoToUrl(link);
				Thread.Sleep(100);
				/*Start Area: Get rendered page height/width and adjust browser accordingly*/
				try
				{
					var javaScriptExecutor = driverWeb as IJavaScriptExecutor;
					var waitPage = new WebDriverWait(driverWeb, TimeSpan.FromSeconds(explicitWaitTime));
					bool readyCondition(IWebDriver webDriver) => (bool)javaScriptExecutor.ExecuteScript("return document.readyState == 'complete'");
					waitPage.Until(readyCondition);

					string HeightScript = "return document.body.scrollHeight";
					long totalHeight1 = (long)javaScriptExecutor.ExecuteScript(HeightScript);
					int totalHeight = (int)totalHeight1;
					if (totalHeight == 0)
					{
						totalHeight = normalHeight + additionalHeight;
					}
					else
					{
						if (totalHeight <= additionalHeightOffset)
						{
							totalHeight += additionalHeight;
						}
					}
					driverWeb.Manage().Window.Size = new Size(totalWidth, totalHeight);
				}
				catch (Exception e)
				{
					_txtSource.TraceEvent(TraceEventType.Error, 0, e.Message);
					_txtSource.TraceEvent(TraceEventType.Error, 0, e.StackTrace);
					_txtSource.Flush();

					driverWeb.Manage().Window.Size = new Size(totalWidth, normalHeight + additionalHeight);
				}
				/*End Area: Get rendered page height/width and adjust browser accordingly*/

				screenshot = ((ITakesScreenshot)driverWeb).GetScreenshot().AsByteArray;

				if (screenshot.Length == 0)
				{
					throw new Exception("Website not responding.");
				}

				driverWeb.Navigate().GoToUrl("about:blank");
				driverWeb.Manage().Window.Size = new Size(totalWidth, normalHeight);
				return screenshot;
			}
		}
		else
		{
			imageDataWeb = returnErrorImage(); //return error image.
		}
		return imageDataWeb;

	}
	catch (Exception e)
	{
		//log exception
		imageDataWeb = doExtractEmptyPage();
		return imageDataWeb;
	}
	finally
	{
		imageDataWeb = null;
	}
}

Redirect website traffic to https with IIS hosting

Securing the traffic coming to your website is of utmost importance. Websites today running without an SSL certificate or without https cannot be trusted. If your application is hosted on IIS, one trick to redirect (not re-write URL) all traffic via https is to create a new Website in IIS and add httpRedirect in your web.config. HTTP Redirection is not available on the default installation of IIS 7 and later.

If your website domain is e.g. https://www.abc.com, this means the IIS binding on port 443 for your website is using host name “www.abc.com”.
The dns requires to be created pointing to the Hosting Application Server. The new Website that you create for redirection should have a port 80 binding with the same domain name.

Also, add the below configuration in your web.config file:

<configuration>
	<system.webServer>
		<httpRedirect enabled="true" destination="https://www.abc.com$Q" exactDestination="true" httpResponseStatus="Permanent" />
	</system.webServer>
</configuration>

The $Q in the destination URL will preserve the Query strings if any. The httpResponseStatus with value Permanent will redirect the traffic to the destination URL with status code 301. Set exactDestination to false if you want to preserve the relative paths during redirects.

TLS version handling with http web request CSharp

Since you’re reading this, probably your C# code broke the connection to the website or a third-party API you’re hitting using the HttpWebRequest. The below code shows one such scenario where my application code broke which is running in .net framework 4.0. The company running the API upgraded their security with the TLS version upgrade to 1.2.

The exception that you’re seeing as below while trying to call GetResponse():

The request was aborted: Could not create SSL/TLS secure channel.

Uri url = new Uri(Link);
HttpWebRequest http = (HttpWebRequest)WebRequest.Create(url.ToString());
HttpWebResponse resp = (HttpWebResponse)http.GetResponse();
returnValue = Convert.ToInt32(resp.StatusCode);

HTTPS relies on a family of lower level security protocol implementations called transport level security (TLS), each using different cryptographic algorithms. Transport Layer Security (TLS) is a cryptographic protocol used to establish a secure communications channel between two systems. Anything that is using TLS standard below TLS 1.2 is considered to be non secure because these older encryption algorithms have been cracked at some point. The TLS standards keep evolving and TLS 1.3 is in the works.

Each .net framework supports TLS version 1.2 in the following ways:

  • .Net 4.5 and above: Add the below line of code before making the web request in your code. Some blogs say .Net 4.6 and above support it by default and no code changes are required but as I tried it myself, it doesn’t work.
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12
  • .Net 4.0: This framework does not support the enumeration as in the latest frameworks, the below line of code helps achieve that:
ServicePointManager.SecurityProtocol = (SecurityProtocolType)3072;

Older frameworks do no support the latest TLS version, so it’s better to upgrade your application. Also, as a good security practice do not use the fallback code as shown below:

System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;

Another good point to consider upgrading your application is to check if Microsoft still supports the .Net framework you’re using.

Show some love for the pit in my PayPal account.