Jump to content


Photo
- - - - -

created a conversion queue for 2.1 and block spammers too


  • Please log in to reply
10 replies to this topic

#1 brianatthebeach

brianatthebeach

    Member

  • Members
  • PipPipPip
  • 97 posts

Posted 18 March 2013 - 06:11 AM

hi all,

I while back I modified the /modules/upload/video.php script to reconvert videos that did not complete the conversion process. I then added a link in avs admin next to each video so that I could click on to reconvert videos. The conversion queue uses the same reconvert script with minor additions.

see here:
http://forums.adultv...light=reconvert

My problem began when I got some very good users of my site posting 10+ videos all at the same time. It would bring my server to a crawl, thus, many videos not only did NOT complete the conversion process but I would have to reboot the server since it would run out of memory and start using swap, thus slowing the whole server down!

So I decided to try this out and it has been working for me so far. I commented out a line in /modules/upload/video.php so that the videos would not convert, they upload, and create a record in the database but they do NOT convert with this line commented out.

Then I modified the /modules/upload/video.php file again so that it would run via a cron job every 15 minutes and convert a single video every 15 minutes.

I needed to add a field to the video table so that the new script would know if the script converted a video already or not. That's pretty much it.

My server has been working like a charm ever since.

If anyone wants to know more specifics please let me know but you need to be familiar with avs script. This may be a little more than a noob should attempt. Also I do not want to log onto anyone's server and be responsible for someone's server. So if you attempt to do this you may need the help of the community as whole.

Also I am hoping someone smarter than I will improve on the methodology that I am using.

I'll attach the modified script if there is interest and go into greater details.

Last thing, all modifications were done on avs 2.1 files. Doubt if this will work on prior versions to 2.0 and may or may not work on avs above than 2.1.

 

UPDATE

I've attached the script.  Rename it to convert_queue.php.

 

Here are the steps to modifying AVS to create conversion queue

 

1)  first you have to create a new column in the video table called 'convertattempt'  type 'char' length = 1 default value  = 0

2) then when you are ready with your script you will populate the existing 'convertattempt'  table with the number 1 so that our cron job does not try to reconvert the videos that you already have live on your site.

3) the cron job as set in cpanel is as below with path to php and then path to the script on your server
/usr/local/bin/php -f /home/yourserveraccountname/public_html/youravsdir/include/convert_queue.php

set your cron job to run as often as you want, it will convert one video at a time.  if your server can convert a video in 5 minutes then you can set the cron job for every 5 minutes.

4) in avs script located at

/modules/upload/video.php comment out the following line

//            run_in_background($cmd.' > '.$lg);

so that the videos do not convert when the user uploads their video

5) fill out your database login and user info in the script attached convert_queue.php (rename the attached convert_queue.blank.php.txt to convert_queue.php)

that should be all to it.

6) you can do a test convert to make sure your script edits work by entering the url below.  after you populate the database field with convert attempt with 1s leave one video with convert attempt set to 0.  then when you test the script using your url it should convert that video that  you set to 0.

http://yourwebsite.c...nvert_queue.php

if no errors and video converts then everything should work and you can start the cron job.

 

7) to block spam from converting enter spammer's UID into the line
    if ( !$errors ) {

 
replace with
 
    if ( !$errors  AND $UID != 'UID1' AND $UID != 'UID2' AND $UID != 'UID3' AND $UID != 'UID4' AND $UID != 'UID5') {
 
where UID1 UID2 UID3 UID4 UID5.... UIDn are the user ids of the spammer's account
 

Thanks,
Brianatthebeach

Attached Files


Edited by brianatthebeach, 16 October 2013 - 05:17 AM.

  • Damien likes this

#2 xtremxpert

xtremxpert

    Junior Member

  • Members
  • Pip
  • 6 posts

Posted 31 May 2013 - 03:43 PM

I would like the reconvert script if you want to share it.
I grabbed a number of videos but after a while the conversion stopped and I am left with a lot of videos in the /vid/ folder while they are already inserted into the database so all I would need would be to re-start their conversion.

#3 theone

theone

    Senior Member

  • Members
  • PipPipPipPip
  • 371 posts

Posted 01 June 2013 - 05:58 PM

I would like the reconvert script if you want to share it.
I grabbed a number of videos but after a while the conversion stopped and I am left with a lot of videos in the /vid/ folder while they are already inserted into the database so all I would need would be to re-start their conversion.


these are two separate issues. a conversion que will not solve your problem.

#4 xtremxpert

xtremxpert

    Junior Member

  • Members
  • Pip
  • 6 posts

Posted 02 June 2013 - 08:16 AM

these are two separate issues. a conversion que will not solve your problem.


I'm well aware of that, that's why I asked for the reconvert script. Also, the conversion queue will solve my problem from now on.

#5 Damien

Damien

    Junior Member

  • Members
  • PipPipPip
  • 78 posts

Posted 02 June 2013 - 03:59 PM

hi all,

I while back I modified the /modules/upload/video.php script to reconvert videos that did not complete the conversion process. I then added a link in avs admin next to each video so that I could click on to reconvert videos. The conversion queue uses the same reconvert script with minor additions.

see here:
http://forums.adultv...light=reconvert

My problem began when I got some very good users of my site posting 10+ videos all at the same time. It would bring my server to a crawl, thus, many videos not only did NOT complete the conversion process but I would have to reboot the server since it would run out of memory and start using swap, thus slowing the whole server down!

So I decided to try this out and it has been working for me so far. I commented out a line in /modules/upload/video.php so that the videos would not convert, only upload, create a record in the database but NOT convert.

Then I modified the /modules/upload/video.php file again so that it would run via a cron job every 15 minutes and convert a single video every 15 minutes.

I needed to add a field to the video table so that the new script would know if the script converted a video already or not. That's pretty much it.

My server has been working like a charm ever since.

If anyone wants to know more specifics please let me know but you need to be familiar with avs script. This may be a little more than a noob should attempt. Also I do not want to log onto anyone's server and be responsible for someone's server. So if you attempt to do this you may need the help of the community as whole.

Also I am hoping someone smarter than I will improve on the methodology that I am using.

I'll attach the modified script if there is interest and go into greater details.

Last thing, all modifications were done on avs 2.1 files. Doubt if this will work on prior versions to 2.0 and may or may not work on avs above than 2.1.

Thanks,
Brianatthebeach


hey mate you still have the script? I need to slow down conversion on my server :)

#6 Damien

Damien

    Junior Member

  • Members
  • PipPipPip
  • 78 posts

Posted 23 June 2013 - 03:45 PM

these are two separate issues. a conversion que will not solve your problem.


His script will help with this, just change value back to 0 in the database and it will reconvert when the cron runs next. Providing you have the original video still (not deleted after convert). It's working very well for me.

#7 Damien

Damien

    Junior Member

  • Members
  • PipPipPip
  • 78 posts

Posted 20 April 2014 - 09:32 AM

small fix for some errors that come up when it can't locate 0 in the table, check before we fetch it

$result2 = mysql_query("SELECT title,UID,active,convertattempt, vdoname,VID FROM $table  WHERE convertattempt = '0'");
$num_rows2 = mysql_num_rows($result2);

$title=mysql_result($result2,0,"title");
$UID = mysql_result($result2,0,"UID");
$vdoname=mysql_result($result2,0,"vdoname");
$active = mysql_result($result2,0,"active");
$convertattempt=mysql_result($result2,0,"convertattempt");
$VID=mysql_result($result2,0,"VID");

replace with

$result2 = mysql_query("SELECT title,UID,active,convertattempt, vdoname,VID FROM $table  WHERE convertattempt = '0'");
if (!$result2 || !mysql_num_rows($result2)) {
    die('Nothing to do.');
}
while ($row = mysql_fetch_array($result2)) {
    $num_rows2 = mysql_num_rows($result2);
    $title=mysql_result($result2,0,"title");
    $UID = mysql_result($result2,0,"UID");
    $vdoname=mysql_result($result2,0,"vdoname");
    $active = mysql_result($result2,0,"active");
    $convertattempt=mysql_result($result2,0,"convertattempt");
    $VID=mysql_result($result2,0,"VID");
}

Edited by Damien, 20 April 2014 - 09:51 AM.


#8 davym123

davym123

    Newbie

  • Members
  • Pip
  • 17 posts

Posted 20 April 2014 - 03:34 PM

Thanks!

 

Trying it out right now! The php script is now running.. but it seems like in the proccess list I cant see me convertion?

Only this:

KFTKhgS.png

is this normal?

 

Thanks again



#9 Damien

Damien

    Junior Member

  • Members
  • PipPipPip
  • 78 posts

Posted 25 April 2014 - 05:18 AM

Thanks!

 

Trying it out right now! The php script is now running.. but it seems like in the proccess list I cant see me convertion?

Only this:

KFTKhgS.png

is this normal?

 

Thanks again

 

you need to set it up to run as a cron job, every 5 minutes or so it hits the php file, looks for any video with convertattemp '0' and runs convert script. You should directly call the file first to see if it works or is connecting to the DB. It won't run if you have the loader.php enter screen running, also.

 

http://some.com/incl...nvert_queue.php

Connected to MySQL OK
Connected to Database OK
Message: Nothing to do.

cron

0,5,10,15,20,25,30,35,40,45,50,55 * * * * /usr/bin/php -f /path/to/public_html/include/convert_queue.php

Edited by Damien, 25 April 2014 - 05:21 AM.


#10 Damien

Damien

    Junior Member

  • Members
  • PipPipPip
  • 78 posts

Posted 20 March 2016 - 12:54 AM

Updated for my PHP7 requirements ;D

<?php
define('_VALID', true);
defined('_VALID') or die('Restricted Access!');
require 'config.php';
require 'function_video.php';
require $config['BASE_DIR'] . '/classes/filter.class.php';
require 'function_global.php';
$basedir            = dirname(dirname(dirname(__FILE__)));

$config['BASE_DIR'] = $basedir;
$config['LOG_DIR']  = '/your/paths/tmp/logs';
$config['VDO_DIR']  = '/your/paths/media/videos/vid';

$host               = 'localhost';
$user               = '*****';
$pass               = '*****';
$db                 = '*****';
$table              = 'video';

mysqli_connect($host, $user, $pass) or die(mysqli_error($conn));
echo "Connected to mysqli - OK<br />\n\n";
mysqli_select_db($conn->_connectionID, $db) or die(mysqli_error($conn));
echo "<b>Database Output:</b><br />\n\n";
$result2 = mysqli_query($conn->_connectionID, "SELECT title,UID,active,convertattempt, vdoname,VID FROM $table WHERE convertattempt = '0'");
if (!$result2 || !mysqli_num_rows($result2)) {
    die('Nothing to convert!');
}
while ($num_rows2 = $result2->fetch_assoc()) {
    $title          = $num_rows2['title'];
    $UID            = $num_rows2['UID'];
    $vdoname        = $num_rows2['vdoname'];
    $active         = $num_rows2['active'];
    $convertattempt = $num_rows2['convertattempt'];
    $VID            = $num_rows2['VID'];
}
echo "title: $title | ID: $UID<br>\n\n";
if ($vdoname == '') {
    $result = mysqli_query($conn->_connectionID, "UPDATE video SET convertattempt='2' WHERE VID = $VID") or die(mysqli_error($conn->_connectionID));
    exit;
}
if ($vdoname != '') {
    if (!$errors) {
        $video_id  = $VID;
        $flvdoname = $video_id . '.mp4';
        $vdo_path  = $config['VDO_DIR'] . '/' . $vdoname;
        function run_in_background($Command, $Priority = 19)
        {
            if ($Priority)
                $PID = shell_exec("nohup nice -n $Priority $Command 2> /dev/null & echo $!");
            else
                $PID = shell_exec("nohup $Command 2> /dev/null & echo $!");
            return ($PID);
        }
        exec($config['mplayer'] . ' -vo null -ao null -frames 0 -identify "' . $config['VDO_DIR'] . '/' . $vdoname . '"', $p);
        while (list($k, $v) = each($p)) {
            if (preg_match("/^ID_.+\=.+/", $v)) {
                $lx              = explode("=", $v);
                $vidinfo[$lx[0]] = $lx[1];
            }
        }
        $duration        = $vidinfo['ID_LENGTH'];
        $height          = $vidinfo['ID_VIDEO_HEIGHT'];
        $width           = $vidinfo['ID_VIDEO_WIDTH'];
        $fps             = $vidinfo['ID_VIDEO_FPS'];
        $id_video_format = $vidinfo['ID_VIDEO_FORMAT'];
        $cgi             = (strpos(php_sapi_name(), 'cgi')) ? 'env -i ' : NULL;
        $cmd             = $cgi . $config['phppath'] . " " . $config['BASE_DIR'] . "/scripts/convert_videos.php" . " " . $vdoname . " " . $video_id . " " . $vdo_path . "";
        log_conversion($config['LOG_DIR'] . '/' . $video_id . '.log', $cmd);
        $lg = $config['LOG_DIR'] . '/' . $video_id . '.log2';
        run_in_background($cmd . ' > ' . $lg);
        $result2 = mysqli_query($conn->_connectionID, "UPDATE $table SET convertattempt='1' WHERE VID = $VID") or die(mysqli_error($conn));
        echo "Video: <b>$title</b> is now being converted.<br>\n\n";
    }
}
?>

Edited by Damien, 20 March 2016 - 12:56 AM.


#11 sextube

sextube

    Senior Member

  • Members
  • PipPipPipPip
  • 233 posts

Posted 15 June 2016 - 09:13 PM

Does this work with 3.3? :)