2020年4月6日月曜日

avgle HLS playlist downloader

Decrypt and download HLS playlist(m3u8) of avgle.com video in browser.
Tested on Firefox + uBlock Origin.

avgleHPD ← bookmarklet to run avgle-hls-playlist-downloader.js

  • the source code is on GitHub. https://github.com/avotoko/avgle-HLS-playlist-downloader
  • tested on Firefox + uBlock Origin. 
  • uBlock Origin default filter is blocking the third party scripts on avgle.com.
    Add the following filter to [My Filters] to allow access to my script.
    @@||avotoko.github.io/avgle-HLS-playlist-downloader/avgle-hls-playlist-downloader.js$script,domain=avgle.com
  • [May 7, 2021] AdGuard Japanese filter prevents the CAPTCHA dialog from appearing on avgle.com, resulting in the video not playing.
    Add the following filter to [My Filters] to show CAPTCHA dialog.
    avgle.com#@#body > div[style*="z-index"]:not([id]):not([class])
  • A Windows batch file to download video using the playlist file is available at the end of this page.

Changes

[v.0.1.8 May 7, 2021] 
- Corresponding to changes in page script behavior.
[v.0.1.7 Jan. 25, 2021] 
- For security reasons, it is no longer possible to modify the content or change the mime type using avglehpdPreDownload. You can still change the file name.
[v.0.1.6 Jan. 25, 2021] 
- Added an interface function avglehpdPreDownload to change the file name, content, and mime type to be downloaded.
[v.0.1.4 Jan. 22, 2021] 
- Corresponded to avgle.com specification changes.
[v.0.1.2 Jul. 16, 2020] 
- Changed the method of getting playlist  to support non-encrypted response.
[v.0.1.1 May. 27, 2020] 
- Fixed a bug that video cannot be played on browser.
[v.0.1.0 Apr. 6, 2020] 
- Initial release

Usage (tested on Firefox + uBlock Origin)

  1. Open the video page in browser. In March 2022 uBO hides the close button.
  2. Run avgleHPD.
  3. Click close button overlapping the video. In March 2022 uBO hides the close button. In that case, click play button.
  4. [Download HLS Playlist] button will appear when the playlist is successfully retrieved and decrypted.
  5. Click the button and download avgle.m3u8.
  6. You can play the video using Streamlink.
    streamlink --http-header Referer=https://avgle.com/ file://location/avgle.m3u8 best
    Or you can download the video.
    streamlink --http-header Referer=https://avgle.com/ file://location/avgle.m3u8 best -o video.ts
    • The url described in the playlist(m3u8 file) retrieved by AvgleHPD becomes inaccessible after a while.
      Try to run Streamlink as soon as possible after retrieving the playlist.
    • You must specify the url instead of the local path of m3u8 file.
      How to convert local path to url: https://en.wikipedia.org/wiki/File_URI_scheme
      [Example for Windows]
       downloaded file path: c:\temp\avgle.m3u8
       url: file:///c:/temp/avgle.m3u8
       command line:
        streamlink --http-header referer=https://avgle.com/ file:///c:/temp/avgle.m3u8 best
  • outputs errors and other information to the console of the browser's developer tools.

Extra

avgleHPD-T ← Bookmarklet to download a playlist(m3u8) file by video title name.

avgleHPD-dl.bat ← Batch file for Windows to download the video itself from avgle.com using playlist(m3u8) file. Requires powershell, streamlink and ffmpeg(optional). Tested on Windows 10.

Changes
[v.0.1.3 Feb. 5, 2021]
- Fixed a bug where ts file name could not be changed from temporary to original when ffmpeg was not available.
- Temporary file name is now created with powershell's Get-Data instead of from the %date% and %time% environment variables, which are formatted differently for each country.
- Improved the output information for troubleshooting.
[v.0.1.2 Feb. 5, 2021]
- Fixed a bug that failed to create a temporary playlist file in some environments.
[v.0.1.1 Jan. 29, 2021]
- Fixed some bugs.
[v.0.1.0 Jan. 29, 2021]
- Initial release

Usage

  1. Download avgleHPD-dl.bat and save it in an appropriate folder.
  2. The batch file you downloaded has been blocked by Windows system. Please go to the file's properties and unblock [Security: This file came from another computer and might be blocked to help protect this computer.] in [General] tab.
  3. Drag and drop the downloaded playlist (m3u8) file onto the avgleHPD-dl.bat icon.
  4. Batch file downloads the video using streamlink to the same folder as the playlist (m3u8) file.
  5. Batch file converts ts file to mp4 if ffmpeg is available.

[NOTE]
Path name (title) containing the character '&' cannot be processed correctly.

15 件のコメント:

  1. Thanks for this, awesome skills!

    返信削除
  2. I want to ask, did avgle encrypt the m3u8 file so we can't see it on console-network?
    And how do you know how to decrypt it?
    Thanks :) I dont even really care about downloading my porn, I just curious about this

    返信削除
  3. It worked.
    Thank you very much now i can fap without internet.

    返信削除
  4. is there an update in the working? seems it doesn't work anymore

    返信削除
    返信
    1. It's working fine for me.

      削除
    2. このコメントは投稿者によって削除されました。

      削除
  5. can you help me?, it's not work

    返信削除
  6. it only works on firefox? because greasemonkey is not available on chrome (i use brave browser btw) and using tampermonkey just says "the userscript is not valid, srry", please help i just want to download this but my peanut brain doesn't understand some stuff https://avgle.com/video/kvoYaYUhaYM/fc2-ppv-1857549-%E6%B5%81%E5%87%BA-re-%E3%82%B9%E3%83%AC%E3%83%B3%E3%83%80%E3%83%BC%E3%81%A7%E5%B7%A8%E4%B9%B3%E3%81%AA%E3%82%BB%E3%83%95%E3%83%AC-%E5%8F%8B%E9%81%94%E3%81%8B%E3%82%89%E5%80%9F%E3%82%8A%E3%81%9F%E5%A5%B3%E3%82%92%E9%83%A8%E5%B1%8B%E3%81%AB%E9%80%A3%E3%82%8C%E8%BE%BC%E3%81%BF%E4%B8%AD%E5%87%BA%E3%81%97-fc2

    返信削除
    返信
    1. I tried to run it as a bookmarklet on google chrome and was able to download the m3u8 file without any problems. Please try to run it as a bookmarklet first.

      削除
    2. You are not writing headers for tampermonkey, are you?
      Try this one. works on firefox + tampermonkey

      // ==UserScript==
      // @name avgleHPD
      // @namespace http://tampermonkey.net/
      // @version 0.1
      // @description try to take over the world!
      // @author You
      // @match https://avgle.com/video/*
      // @icon https://www.google.com/s2/favicons?domain=avgle.com
      // @grant none
      // ==/UserScript==

      (function(url,e){e=document.createElement("script");e.src=url;document.body.appendChild(e);})("https://avotoko.github.io/avgle-HLS-playlist-downloader/avgle-hls-playlist-downloader.js");

      削除
  7. wow you are a life saver! thank you very much!

    返信削除
  8. But for a little feedback, sometimes this batch file is stucked when we download a file, so we must press enter, and it will continue to download. i hope you can fix this bug.

    返信削除
  9. Batch file argument %1: D:\Data\avgle-HLS-playlist-downloader-master\avgle-HLS-playlist-downloader-master\avgle.m3u8
    Playlist Path: D:\Data\avgle-HLS-playlist-downloader-master\avgle-HLS-playlist-downloader-master\avgle.m3u8
    TempName: avgle-20211101-17103138

    D:\Data\avgle-HLS-playlist-downloader-master\avgle-HLS-playlist-downloader-master>copy "D:\Data\avgle-HLS-playlist-downloader-master\avgle-HLS-playlist-downloader-master\avgle.m3u8" "D:\Data\avgle-HLS-playlist-downloader-master\avgle-HLS-playlist-downloader-master\avgle-20211101-17103138.m3u8"
    1 file(s) copied.

    D:\Data\avgle-HLS-playlist-downloader-master\avgle-HLS-playlist-downloader-master>streamlink --http-header referer=https://avgle.com/ file:///D:/Data/avgle-HLS-playlist-downloader-master/avgle-HLS-playlist-downloader-master/avgle-20211101-17103138.m3u8 best -o "D:\Data\avgle-HLS-playlist-downloader-master\avgle-HLS-playlist-downloader-master\avgle-20211101-17103138.ts"
    [cli][info] Found matching plugin hls for URL file:///D:/Data/avgle-HLS-playlist-downloader-master/avgle-HLS-playlist-downloader-master/avgle-20211101-17103138.m3u8
    [cli][info] Available streams: live (worst, best)
    [cli][info] Opening stream: live (hls)
    [download][..le-20211101-17103138.ts] Written 139.5 MB (1m43s @ 1.1 MB/s) [stream.hls][error] Failed to fetch segment 180: Unable to open URL: https://ip256332056.cdn.qooqlevideo.com/key=3e4KLPrXesQhvGFDT2vh-w,s=,end=1635766905,limit=1/data=1635766905/state=YX+8noyr/referer=force,.avgle.com/reftag=56109644/media=hlsA/ssd4/177/3/267349553.mp4/seg-180-v1-a1.ts (403 Client Error: Forbidden for url: https://ip256332056.cdn.qooqlevideo.com/key=3e4KLPrXesQhvGFDT2vh-w,s=,end=1635766905,limit=1/data=1635766905/state=YX+8noyr/referer=force,.avgle.com/reftag=56109644/media=hlsA/ssd4/177/3/267349553.mp4/seg-180-v1-a1.ts)
    [download][..le-20211101-17103138.ts] Written 351.4 MB (4m8s @ 1.6 MB/s)

    i got this error

    返信削除
  10. According to the error information provided, the m3u8 files are being retrieved correctly, so avgleHPD is working properly.
    The problem is in the communication between streamlink and the server. This is not my work.

    返信削除