driver.execute_script("var replaced = $('body').html().replace(/(®)/g,''); $('body').html(replaced);")
However, at some point recently this no longer works, giving the following error:
JavascriptException: Message: SecurityError: The operation is insecure.
Presumably, when updating Selenium, Firefox, or geckodriver, this function broke – unless there is something on the web page that I crawl (which is not mine) that has changed, that is making the script impossible to execute (I don't know what on the web page possibly would cause this).
What could be the reason behind this issue, and is it possible to "override" the SecurityError and execute the script?
Have you tried anything like this?
Go to about:Config and verify that dom.serviceWorkers.enabled is set to true
Then, go to about:preferences#privacy and ensure that you have unchecked Delete cookies and site data when Firefox is closed
Seems like the script is trying to update/replace the content - $('body').html(replaced); Maybe after updating it is trying to download some content due to which the error SecurityError: The operation is insecure is thrown.
To triangulate/debug this issue, you may breakdown the script execution and check on which statement the exception occurs -
driver.execute_script("var replaced = $('body').html().replace(/(®)/g,'');")
driver.execute_script("$('body').html(replaced);")
Also, this behaviour does not seem to be with Selenium/GeckoDriver itself. But with the Javascript/JQuery code being executed.
I found these two StackOverflow questions around JQuery + above error which might point you in the right direction -
Link 1
Link 2
Link 3 (this is as mentioned in the above comment)
Here you may try to set pref via Selenium Capabilities to set the values for dom.serviceWorkers.enabled
Will update this answer if I found some other way to handle this behaviour
Thanks to @Xtraterrestrial for providing links in their answer that helped me solve the issue. I have awarded my bounty.
For clarity, I am here providing my particular solution.
As described in one of the linked answers, the SecurityError arises apparently because the JavaScript attempts to modify insecure elements on the web page, such as <input>.
Using the code from this answer instead of the original JQuery used, the script now looks for TextNodes and modifies only those.
var replaceTextInNode = function(parentNode){
for(var i = parentNode.childNodes.length-1; i >= 0; i--){
var node = parentNode.childNodes[i];
// Make sure this is a text node
if(node.nodeType == Element.TEXT_NODE){
node.textContent = node.textContent.replace(/(®)/g,'')
} else if(node.nodeType == Element.ELEMENT_NODE){
// Check this node's child nodes for text nodes to act on
replaceTextInNode(node);
replaceTextInNode(document.body);
Implemented in Python for Selenium as:
driver.execute_script("var replaceTextInNode = function(parentNode){for(var i = parentNode.childNodes.length-1; i >= 0; i--){var node = parentNode.childNodes[i]; if(node.nodeType == Element.TEXT_NODE){node.textContent = node.textContent.replace(/(®)/g,'')} else if(node.nodeType == Element.ELEMENT_NODE){replaceTextInNode(node); }}}; replaceTextInNode(document.body);")
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
