Jump to content
Tuts 4 You

How to fetch infos and output them?


LCF-AT

Recommended Posts

Hi guys,

I would like to ask you whether anyone of you could help to find / show some examples how to fetch data/infos from any web traffic via script (TamperMonkey / GM) or maybe any extension if there are etc. So this time I wanna read specific URL/s when they gets loaded in browser (html,JS,JSON,etc extension) to find a info and output as in browser itself as button or Link or something when found. Problem is to find any examples about it how to do that. So in case of a html webpage I could use something like "document.getElementById" etc but what is in case of other files which are not html files and using such tags? What about JS scripts itself or JSON files or just blank text files etc. How to check them? Maybe you can explain it a little or showing me simple examples I could try out. Thank you.

greetz

Link to comment
Share on other sites

Hi NOP,

hmm  not sure about that extension whether its something for me. So in my case I just want to read some URL when it gets loaded in browser and then checking for parameter X and showing the results in my browser window as button or string or something. Pretty simple task when doing it manually but here I want to automate it by using / creating a small simple script you know. I'am looking for a example to read / print something from html page and from a JSON file and maybe other files too. How to read JSON file / find specific JSON parameter like "NAME: HERE_RESULT" and reading the result out and print it in browser (button / Copy etc). This should happen on fly. Found some info about a GM_getValue() but this seems only to work when using before the GM_setValue().

Maybe another example. Lets say I load Tuts4you and its loading diffrent stuff.

1. Task: Read something from webpage itself from script tags. Below a short part. Now I want to read the paramter like "links_external: " whether it can be found and in case of YES I want to read the set value which is "true" and this I wanna output in browser window as button or string or anything. How to do this?

<script type='text/javascript'>

	disableNotificationSounds: true,
	useCompiledFiles: true,
	links_external: true,

</script>

2. Task: How to read from this span tag below? Just wanna output the string >Very important ! Please read.....<.

<div class='ipsType_richText ipsDataItem_meta ipsContained' data-controller='core.front.core.lightboxedImages' >
<p>
	<span style="color:#FF0000;">Very important! Please read before sign up and posting... </span>
</p>

</div>

3. Task: Reading from JSON when it gets loaded in browser. If you go on YT for example and you do scroll on  the mainpage down then the browser will send a POST request with JSON content and it gets JSON content as answer back and now when this happens I would like to search for an entry parameter etc and reading the results of it and output this as I told above already. Just simple tasks but how to make a script for that? Just wanna make a script because it can check this when it happens in browser on fly without to request the URL/s by itself etc. Just need to find some examples how I could do this or anything.

greetz

Link to comment
Share on other sites

Your wishes are non-standard and most likely not the optimal way to solve whatever problem you're trying to solve. Are you really sure that javascript inside the browser is the right way?

 

For #1 and #2 - getElementsByTagName (https://developer.mozilla.org/en-US/docs/Web/API/Element/getElementsByTagName) and/or querySelectorAll (https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelectorAll) should do the trick.

https://developer.mozilla.org/en-US/docs/Web/API/Document_object_model/Locating_DOM_elements_using_selectors is a good starting point and then you can google for more specific examples.
Like this: https://stackoverflow.com/questions/51807245/how-to-get-list-of-scripts-in-html-document-head

#3 - script running inside the main web page will likely have troubles accessing data that are loaded later from some other site via a POST request. If really desired, it might be doable via some TamperMonkey/GreaseMonkey script but I have no idea how exactly. Neverthelese, please see the first paragraph.

 

  • Like 2
Link to comment
Share on other sites

Hi kao,

not sure whether JS is the right way but at the moment I don't see any other methods I could use to check & grab datas on fly inside of the browser itself. In my case I don't want to write a extern app to load specific webpages and handle response / request paramters like cookies & Co you know. So when I'am using a TM / GM script then it can or should do those steps for me directly on fly when the browser does read the webpages XY by itself. Also in my case I just want to do pure simple find & output stuff and no complex things or something. Also same with JSON files. How to do that?

greetz

Link to comment
Share on other sites

One more thing. Is there maybe any extension out there what could check for JSON files when they was loaded on loading a webpage? Lets say I load tuts4you and during this process there are maybe loaded 4 JSON files (content-type: application/json) and now I want to get a info about that. Lets say a icon of extension what shows now a value of 4 = found 4 JSON files (checking content-type). Now I wanna press on the icon to get a list of all 4 URLs with that JSONs on this webpage and now I wanna press on any of them to open this JSON file into new tab nice and clean formated. Something like that. I know Firefox has a own JSON formater but Chrome / Brave not. Is there already any extension or anything what could do that maybe?

greetz

Link to comment
Share on other sites

I don't know of any extensions but it is probably doable with gm/tm

You can do a simple text search like this

if (/links_external: false/i.test (document.body.innerHTML) )
	alert ("links_external: false");
else if (/links_external: true/i.test (document.body.innerHTML) )
	alert ("links_external: true");

or a regex search

var m = document.body.innerHTML.match(/(links_external: \w+)/i);
if (m)
  alert(m[0])

or you can loop through tags (example will show all png links)

var links = document.getElementsByTagName( 'link' );
var element;
var str='';
for ( var i = 0; i < links.length; i++ )
{
	element = links[ i ];
	if (element.href && element.href.match(/(.+\.png)/i))
		str += element.href + '\n';
}
if(str)
	alert(str);

my gm/tm scripting isn't great but that should be enough for you to get started on most of what you want to achieve. Obviously the page alert can be changed to console.log or whatever

  • Like 2
Link to comment
Share on other sites

Hi NOP.

thanks, so can I also test it on fly what you wrote on that w3school or so? But this is only for html sites and nothing more. What about checking other files which are not html and using such tags? Would like to check JSON files for example.

greetz

Link to comment
Share on other sites

You would add that as a gm/tm script

You could try on w3school to test but you would have to add the above javascript in javascript tags in the same html, it's probably easier to test using gm/tm

I have no idea how to check a loaded json file sorry but if you can't figure it out then you can at least show a list of them so you could load them separately

  • Like 1
Link to comment
Share on other sites

Hi NOP,

what do you mean with showing a list of them?

Question: Is there a function in FireFox & Chrome to read the loaded URL/s when they get loaded in browser (like I can see all traffic in network tab or fiddler etc) ? If yes, is there a way to log the entire request parameters & possibly the content of it? Is there a way to repeat the single URL request with all parameters in a new tab from a extension or script? I'am just looking a methods to grab URLs / content on fly / repeat requests if needed to show the content of it. Yes, I can all do with network tool itself so there is all logged and I can see & check all etc but this manually way sucks and I would like to automate this process.

greetz

Link to comment
Share on other sites

I mean you can get the script to search for whatever links in the html and then output you a list of them all

I don't know of an extension but Fiddler will do everything you need and a lot more

  • Like 1
Link to comment
Share on other sites

Hi NOP,

I think we turn in a circle around. :) Maybe its working just for html sites to check for tags and do some search things etc but I'am looking for some more specific searching also outside from html files. Just wonder why this is not doable with TM / GM scripts you know.

greetz

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...