Scraping web with java and downloading a video

Refresh

5 weeks ago

Views

4 time

0

I'm trying to scraping this 9gag link

I tried using JSoup to get this HTML tag for taking the source link and download the video directly.

I tried with this code

    public static void main(String[] args) throws IOException {
        Response response= Jsoup.connect("https://9gag.com/gag/a2ZG6Yd")
                   .ignoreContentType(true)
                   .userAgent("Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0")  
                   .referrer("https://www.facebook.com/")
                   .timeout(12000) 
                   .followRedirects(true)
                   .execute();

        Document doc = response.parse();
        System.out.println(doc.getElementsByTag("video"));
    }

but I get nothing

I tried then this

    public static void main(String[] args) throws IOException {
        Response response= Jsoup.connect("https://9gag.com/gag/a2ZG6Yd")
                   .ignoreContentType(true)
                   .userAgent("Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0")  
                   .referrer("https://www.facebook.com/")
                   .timeout(12000) 
                   .followRedirects(true)
                   .execute();

        Document doc = response.parse();
        System.out.println(doc.getAllElements());
    }

and I noticed that in the HTML there is not the tag I'm looking for, as if the page is loaded dynamically and the tag "video" is not loaded yet

What could I do? Thank you all 😊

0 answers