Posts Tagged ‘recorder’

Setting up an automated ad monitoring service for TV

Saturday, May 26th, 2012

So you want to set up your own automated advertisement monitoring for some TV channels? And you probably have an idea how to sell the reports from the whole system? Let me try to explain one of the possible ways of doing it.

Overview

Advertisement monitoring system isn’t so complicated, but it isn’t simple either. You’ll need computers, people, and some kind of service to automatically track advertisements that are spotted once.

Recording

For starters, you have to be able to record all your needed TV channels. Depending on the TV system used in your country, you’ll have several options for it. From our shop, we can solve recording for analog tv, DVB-T, DVB-S, IPTV. In any case, if you can get composite video signal from your set-top box, you will be able to record it with VideoPhill Recorder.

Storing and archiving

Recorded broadcast should go to some storage, depending on the number of days that you want your broadcast archive to be available. nike air max 1 pas cher To calculate how much storage space you will need for it, you can use this on-line calculator.

Clipping and tagging

So now we have recordings of the TV broadcast. Fjallraven Kanken Big UK Next step is to form a team of people who will find and tag the first occurrence of an advertisement. Nike Air Max 2016 Goedkoop Number of people and workstations required for the job depends on many factors:

  • number of channels monitored
  • channel ‘difficulty’ (how easy is to find commercials on the channel)
  • number of shifts that people will do

In short, you’ll need some way of accessing the archive and clipping the portions of it in order to have clips of advertisements extracted and prepared for automated archive search.

One possible way of doing the job is by using VideoPhill Player application. fjallraven kanken To see it in action, please see video below…

Automated search

Almost there… Now, you have your archived broadcast, and you have your clip library. To find all of the occurrences of all clips on all your channels, you’ll simply pass whole archive and clip library to a PlayKontrol Service and get your results. Vêtements Armani Pas Cher Results can be in any format that you require, such as text, excel, PDF, XML, and so on.

Producing reports for your customers

Really final component of the system (apart from selling the reports) is a team of people who will use raw data that PlayKontrol will provide and produce nice reports for your customers.

Tags: , , , , , ,
Posted in marketing, tech | 1 Comment »

Creating a small (hopefully usable) utility: File Deleter

Friday, May 11th, 2012

When you are in media monitoring, you have TONS of files. nike homme For example, look at this:

Multitude of files, StreamSink archive

Another bunch of files, created by PlayKontrol

Every recorder and logger and input produces a number of files on your system. Of course, each application such as VideoPhill Recorder or StreamSink have the option for deleting a files after they expire (one month for example), but what if you have other way of gathering information (media, metadata, something) that won’t go away by itself? I have several such data sources, so I opted to create a MultiPurposeHighlyVersatile FileDeleter Application. sac fjallraven kanken pas cher I’ll probably find a better name later, for now lets call it ‘deleter’ for short.

The Beginning

The application to delete files must be such a triviality, I can surely open Visual Studio and start to code immediately. Asics Gel Lyte 3 Well, not really. Adidas Dames In my head, that app will do every kind of deleting, so let’s not get hasty, and let’s do it by the numbers.

First, a short paragraph of text that will describe the vision, the problem that we try to solve with the app, in few simple words. That is the root of our development, and we’ll revisit it several times during the course of the development.

Vision:

‘Deleter’ should able to free the hard drive of staled files (files older than some period) and keep the level of hard drive space at some pre-determined minimum.

Here, it’s simple enough that I can remember it, and I’ll be able to descend down from it and create the next step.

The Next Step

For me, the next step (let’s say in this particular case) would be to try and see what ‘features’ does the app have. Nike Pas Cher The only way it works for me is to create a mock of the application UI and write down the things that aren’t visible from the UI itself. Comprar Nike Air Max Since this UI won’t do anything but gather some kind of parameters that will define behavior of the app, it will be a simple one, and it will be possible to fit it nicely on one screen.

For the sketch I’ll use Visual Studio, because I’m most comfortable with it. If it wasn’t my everyday tool, I’ll probably use some application such as MockupScreens, which is completely trivialized app sketching gadget with powerful analyst features.

The process of defining the UI and writing down requirements took some time, I repeatedly added something to UI, then to the list below, until I had much clearer picture what I’m actually trying to do.

Features:

  • it should have ability to delete only certain files (defined by ‘mask’ such as *.mp3)
  • it should be able to delete file by their age
  • it should be flexible in determining the AGE of the file:
    • various dates in file properties: created, modified, accessed
    • by parsing the file name of the file
  • it should be able to delete from a multiple directories
  • it should be able to either scan directory as a flat or dig into subdirectories
  • it should be able to delete files by criteria other than age
    • files should be deleted if their total size exceeds some defined size
      • in that case, other files should be taken into account, again by mask
    • files should be deleted if minimum free drive space is less then some defined size
    • file size
  • when deleting by criteria other than file age, specify which files should be first to go
  • should be able to support multiple parameter sets at one time
  • should run periodically in predetermined intervals
  • should be able to load and save profiles
    • profiles should have names
  • should disappear to tray when minimized
  • should have lowest possible process priority
And here’s the screen:

Mock of the Deleter UI used to define and refine the requirements

As you look at the UI mock, you’ll see some mnemonic tricks that I use to display various options, for example:

  • I filled the textboxes to provide even more context to the developer (myself with another cap, in this case)
  • I added vertical scrollbars even if text in multi-line textboxes isn’t overflowing, to suggest that there might be more entries
  • for multiple choice options I deliberately didn’t use combobox (pull down menu) – I used radio button to again provide visual clues to various options without need for interaction with the mock

From Here…

I’ll let it rest for now, and tomorrow I’ll try to see if I can further nail down the requirements for the app. From there, when I get a good feeling that this is something I’m comfortable with, I’ll create a object interface that will contain all the options from the screen above. While doing that, I’ll probably update requirements and the UI itself, maybe even revisit The Mighty Vision above.

BTW, it took me about 2 hours to do both the article and the work.

Tags: , , , , ,
Posted in tech | 8 Comments »

Capturing and archiving of DVB-T signal

Saturday, May 28th, 2011

No matter if it’s for compliance recording so you will capture and save your own broadcast, or you are doing media monitoring and you would like to capture multiple signals of the air, you have some interesting choices here.

Let’s explore in detail your options on the subject, whether it’s one channel or multiple channel recording.

One channel DVB-T recorder

Recording of one channel is simple no matter how you choose to record it. Let me present two main options here for you, so you could see what is most applicable in your situation.

Simplest way of recording would be to have one set top box for DVB-T, and use it to send composite signal into the computer via the Osprey 210 card. It is the most robust solution, but it has some (serious) drawbacks:

  • cheap DVB-T tuners can ‘lock’ and freeze the picture
  • low quality tuners can also de-sync audio and video with time – and you need 24/7 operation here
  • you’ll need extra power connector for the set top box
  • STB-s are producing extra heat

Alternative way of recording is to use DVB-T card such as Asus MyCinema-ES3-110, use software such as TubeSink to tune on a frequency and extract the channel required from it (this is called DEMUX-ing) and forward the extracted channel to the VideoPhill Recorder for further processing (recording, streaming, …).

BTW, TubeSink mentioned above can be used even without VideoPhill Recorder, as it DEMUXes the channels and can forward them to any computer on your network as an UDP Transport Stream that can be playable with VLC. Nike Air Max 2017 Heren grijs It you want to use it for non-commercial purposes, download it from here.

So in the case on 1 channel DVB-T recording, I would say that it remains uncertain whether to use external set top box with Osprey capture card, or go with pure software solution and some simple of-the-shelf DVB-T tuner.

But in case of…

Multiple channels DVB-T recording facility

Same options are available at multiple channel recording facilities – but here is the catch. As you might probably know, multiple DVB-T channels are packed and are transmitted at one frequency and that is called multiplexing. The carrier for the channels that are transmitted is called MULTIPLEX (MUX for short). nike air max 2017 pas cher In several occasions it has 4 channels, and sometimes it can have as much as 16 or more channels.

Current recommended recorder density (channels per machine) is 4. Adidas Gazelle Heren One machine packed with Osprey 460e will do 4 channels just fine.

So, let’s say that we need 16 channels and they are scattered across 3 MUX-es (we have such situation here in Zagreb). Using a conventional method (I would say that having 16 STBs is conventional, as bizarre as it seems) you’ll need the:

  • 4 recording servers
  • 4 Osprey 460e cards
  • 16 DVB-T set top boxes
  • PLENTY of mains outlets
  • some kind of distribution to have the signal distributed to all 16STBs

Since you see where I’m coming to, let me suggest the following; let’s use TubeSink to control 3 tuners in TWO MACHINEs, and save on 4 Ospreys and 2 PCs, and the rest of the unnecessary equipment.

We’ll put 2 tuners into one machine, and one tuner in the second machine. kanken fjallraven If the channel per MUX distribution is such that each machine has it’s 8 channels, fine. Nike Air Huarache Heren If not, we’ll instruct TubeSink to forward the Transport Stream to ANOTHER machine and that machine will perform recording. In that way, load will be completely balanced between two machines, and you’ll have your 16 channels recorder in a nice and compact fashion.

Even more compact?

Yes, it can go even further. There are dual DVB-T tuners such as WinTV-HVR-2200 that can provide tuning to two frequencies at once, and with it, you could record as much channels there are in two MUXes at one machine. Nike Air Max 2017 Homme Today, even desktop processors such as i7 can encode 8 channels of video in real time. So, with proper CPU (or multiple CPUs on server computers) – even 16 channels could be encoded in one compact 2U rack mounted unit.

However… (serious problem)

Using PC based DVB-T cards will only work with free to air channels.

Tags: , , , , ,
Posted in tech | 1 Comment »

SD-SDI compliance recording

Saturday, May 28th, 2011

I will try to write something about creating the archive for the compliance recording purposes from the SD-SDI source. This post is in a response from an repeated inquiries about system that would create such an archive.

Compliance recording in general

Just to remind us – compliance recording serves only one purpose – to prove or disprove that something went on the air at some time. That is the first and the last thing that this technology is used for. nike air max 2017 goedkoop There is no any requirement on the content of the signal – it just have to be good enough for someone to see basic stuff that’s in there .

This one business requirement is countered from the other side with need to create the archiving system to be as affordable as possible. And the cost of the system, if we count out the software involved (system and application part) is highly dependent on the:

  • picture quality
  • number of channels recorded
  • number of possible outputs needed
  • and days of the archive that should be kept

Picture (ie. video) quality is directly proportional to the BITRATE of the video that is recorded. More bitrate, more apparent quality, both in picture resolution, object motion, and so on. cheap fjallraven kanken uk For example, most recorded videos that you might have on your computer are in between 700 and 1000 kbit and are recorded with DivX or similar encoder. Watching a movie at 1000 kbit bitrate is highly enjoyable, and everything above that is required ONLY for HD or HD-Ready content.

Compliance recording means recording what’s on the air

Many people forget that when you do compliance recording, you have to record what was coming out of your transmitter array, not the final that you are broadcasting. In case your link went down, or your output HF amplifier burned, your endpoint picture will be NOTHING, and if you are recording your output, there will be a great discrepancy in your logs.

So suggestion: drop the thinking about recording SD-SDI, and try to record what comes back from the air, in the form of the analog or DVB-T signal. That is the REAL broadcast that your viewers see.

The conflict of interest

We all want stuff to be as cheap as possible. In order to build a recording system that is affordable, we have to balance between various things, but here I’ll try to explore the differences when SD-SDI signal capture is required.

I will postulate this input parameters for the archive that will serve as example here:

  • video bitrate: 512 kbit
  • audio bitrate: 64 kbit
  • archive duration (number of days to keep from today): 92 (3 months)
  • number of channels that needs recording: 4
  • required outputs: both WMV (Windows Media Video) and h.264 (at the same time)
  • recording resolution: 3/4 of PAL D1 picture size so: 540×432

In order to calculate hard drive space requirements for this here, I’ll use the online bitrate calculator here on this blog.

Table below taken from the calculator itself.

Video Bitrate Audio Bitrate Total Bitrate Days Channels Disk Space
512 kbit/s 64 kbit/s 2304 kbit/s 92 4 2183.20 GB

So we need 2200GB of net drive space. In order to provide that kind of space with some fault tolerance, I would recommend having two drives such as WD CAVIAR GREEN 2500GB connected in a MIRROR VOLUME (RAID1). There is no need for additional system drive, and almost all new motherboards can provide on-board implementation of RAID1 – mirroring.

I’m kind of beating around the bush here – because main point of this article should be to provide you with a choice of whether to use original SD-SDI signal or convert to composite signal (or use the signal that comes back).

Osprey vs DeckLink

The choice that lies before us is either to use Osprey 460e card, or a DeckLink Quad card. List price for the Osprey is about $1200, and two DeckLink Quad card is about $1000. So, the first impression is that you’ll go cheaper with DeckLink. But…

So far, the Osprey has proven itself to be absolute master of video capture. Nike France Every system deployed so far didn’t have any problems with the card whatsoever. Tennis Nike France You plug it in, install drivers, and it works. adidas shoes uk And with Osprey 460e, you’ll use only ONE PCI-e slot. Fjallraven Kanken Big Other slot, if there is any, will be used by the graphics card.

Same thing about the slots applies to DeckLink as well. However, there is “now shipping” image below the DeckLink Quad card name, and it implies something – it is red-hot new.

Tags: , , , ,
Posted in tech | No Comments »

How to reduce hard drive fragmentation

Tuesday, May 17th, 2011

The topic of drive fragmentation might be a little out in this days, but since I spent great deal of my youth watching PC Tools defragment my drive in a graphically pleasing fashion, I am inclined to think that drive fragmentation (when excessive) can severely reduce both computer performance and hard drive life.

As this might be true for the common day-to-day user, it is particularly true for corporate/enterprises that do need their data to be:

  • accessible,
  • quickly accessible,
  • accessible for a long time

In a common computer use scenario, most of the files are there for computer to read an use, either as software that has to be loaded into memory, or documents that have to be shown to the user. Writing to the hard drive is uncommon operation (when you put it against the number of reads) and thus the drive fragmentation however present is in fact easily ignored.

Continuous stream recording, enter…

In my business (my clients businesses’ to be exact) the hard drives are working in opposite. They WRITE all the time, and read only on occasions. And the problem that will surely lead to fragmentation is that in most situations they need to write MULTIPLE long files continuously. adidas 2017 pas cher Let me try to explain what, first from the aspect of why, then move to what…

When either running VideoPhill Recorder for recording video, or using StreamSink to record internet media streams, in most cases user has MULTIPLE channels recorded on one computer. Files that are created by that recording are commonly created at one time (all of them) and are grown continuously until closed. Since Windows is, as it is now, an operating system that can’t reserve drive space in advance (maybe it can, but software doesn’t know how long the files would be) the space for them will be allocated as the time goes by. If we have 4 files that are written slowly but concurrently (and are grown at the same time), we’ll certainly have the following situation on the hard drive (I’m talking ONLY about the data that is stored here, and am simplifying physical hard drive storage as a continuous slate):

file1_block1
file2_block1
file3_block1
file4_block1
file1_block2
file2_block2
file3_block2
file4_block2
.
.
.
file1_blockN
file2_blockN
file3_blockN
file4_blockN

That means fragmentation. fjallraven kanken soldes File isn’t in continuous blocks, but is scattered in evenly and can’t be read sequentially from the hard drive. fjallraven kanken rugzakken You might be lucky and your blocks could be scattered in a way that sectors on the drive will be adjacent and this won’t pose a problem, but what are the chances? :)

And when file1 gets deleted, what remains on the hard drive? A blocks filled with nothing, left there for other files to fill them. New files will try to fill them, and the drive will soon be completely jumbled. It will all be hidden from you by the OS, but still, OS will have to deal with it.

And that is the story of 4 channels. What about situation when you have 60 channels recorded on one machine (I’m talking about internet stream recording, of course). Such an archive could be found here: http://access.streamsink.com/archive/

If you aren’t convinced that this really IS a problem, you can stop reading now.

Rescue #1 – Drive Partitioning

It is feasible in situations where there is low number of channels that needs to be recorded. If you have 4 channels, you’ll create 4 partitions, and each partition will have nice continuous files written to it. Done.

However, you can’t have 50 partitions on one drive and get away with it.

Rescue #2 – Queued File Moving

Other solution for large number of channels presents itself in a form of a temporary partition for initial file recording, and then moving out the files to their permanent location later, but ONE FILE at a time, in a queue.

Queued Moving of Files in StreamSink

This is implemented in StreamSink, and it even has an ability to throttle data rate when moving the files to another drive. adidas schoenen Only thing that is of a problem here is wasting of a temporary hard drive, because it gets beaten by fragmentation.

Rescue #3 – Using RAM Drive on Method #2

While I was writing the article about NAS, thought flashed across my mind – can we avoid writing to the temporary drive and reduce the load ONCE more?

Yes, we can. Mochilas Kanken No.2 I know that RAM Drives are also out of fashion, but here one will come handy. It’s the shame that support for it isn’t included in the system already, so with little googling I found this: http://www.ltr-data.se/opencode.html/#ImDisk

I installed it on the testing server, re-configured the application to use new temporary folder, and from now on, it runs so smooth I can’t hear it anymore :)

Some technical stuff:

  • in this instance, I am currently recording 62 channels and cumulative rate for it is around 5 megabit/second
  • my files have duration of 5 minutes, which means that recorded chunks are closed and moved to permanent storage every 5 minutes
  • during those 5 minutes, each file will grow so much that the whole content for those 5 minutes won’t get over 200megabytes
  • I created 512 megabyte ram drive, just to be safe

Conclusion

Take care of your hard drive, and don’t dismiss old-techs such as RAM Drives just yet.

If I was about to implement this on an application level, I would have to spend a great deal of time, and some media types won’t even be possible to implement – Windows Media for example, writes to disk or to other places if you employ magic…

Tags: , , ,
Posted in tech | 8 Comments »

Having NAS is great (or is it?)

Tuesday, May 17th, 2011

During the years I had many deliberations over the fact if either NAS would be used or it wouldn’t be used for the video archives created by video logger system such as VideoPhill Recorder. air max pas cher At first, I was firm believer in one methodology, then completely turned my side to the other, and now, … Well, read on, and I’ll take you through it.

Stakes

On the one side of the stake set we have actual requirements, and on the other there are considerations. Actual requirement are sometimes hard to pinpoint at first, but they always come out sooner or later.

So, let me list possible requirements that might be in effect here…

Common low level requirements

Storage for video recording (for logging purposes) needs to have following abilities:

  • low but constant and sequential write rate – data rate for 4 channels are as low as 5mbit per second (500kbytes/sec) but is CONSTANT and SEQUENTIAL – there won’t be much stress for the hard drive because of constant seeking
  • high durability over time – what gets written once, has to be there. Nike Air Max TN Homme It should survive single drive failure
  • reading isn’t common, but when done, it has to be sustainable, but again at low data-rate and great predictability (it usually is sequential)
From everything above, I can guess that any data storage expert would read RAID 5 and won’t allow you to create anything else for the video archive storage.

Archive duration scalability

The archive duration is directly proportional with the hard drive space that is available. To determine what kind of hard drive space you need for your first installation, you can use on-line hard drive size estimator calculator that I created right for this blog.

If you plan extend the duration of your archive one day, you have this requirement, and you have to plan for it. Having the storage at one place can simplify the adding of the drive space, but can also completely block it.

Let’s say that you have the archive of 40 channels that span 92 days (3 months). And let’s say that you decided to use 1mbit video with 128 kbit audio for the archive. By using the calculator above, you’ll find out that you have 42 terabytes of storage already in place. Even at this date, that kind of storage set in one place is kind-of-a challenge to build.

If you have already invested in 42 TB storage system, and have foresight to plan for an upgrade to say its double size for it, you are in luck. But, say that after a few more months (just 3) your management decides to expand the requirement to 12 months of storage. Wow. nike air max 90 pas cher Now, you have to have 127 TB total. If the current system will hold that much drive space, again you are in luck, however – say it doesn’t. Nike Air Max 2016 Heren blauw Your options are:

  • add one more to the chain
  • create a bigger unit, copy everything to it, scrape the current one
I’ll stop my train of thought here, and leave you only with few things to think about before I go on with other requirements: who needs used 82 TB system (if you want to sell it), do you know how much it is to COPY 82 TB even at extreme network speeds, adding one more will break the ‘all in one place’ requirement, …

Having it all in one place (i.e. for web publishing)

If you need everything in one place for the publishing, then this is a solid requirement. Scarpe Adidas Web server will have the content on its local hard drives, and it will publish it smoothly.

But, is that really a requirement? I must admit that I didn’t see web server that properly served files from the network locations (despite the thing that there are option to do that), but I’m sure that IIS and Windows Server gurus will be able to shut me down and say that this is normal thing that is done routinely. So, if we know that each channel recorder has its own directory ANYWAY, what’s the use of having

c:\archive\channel1
c:\archive\channel2
c:\archive\channel3

instead of

\\rec1\channel1
\\rec1\channel2
\\rec2\channel3

Reducing single point of failure

This requirement is very common, and having NAS system as a ‘point’ it brings us that having the NAS leads us to having single point of failure. I understand that there are multiple redundancies that could be installed into the system, such as RAID 5, or obscene configurations such as RAID 1+5. Note: for later, it seems that the article author has a same opinion on it as me:

Recommended Uses: Critical applications requiring very high fault tolerance. In my opinion, if you get to the point of needing this much fault tolerance this badly, you should be looking beyond RAID to remote mirroring, clustering or other redundant server setups; RAID 10 provides most of the benefits with better performance and lower cost. Not widely implemented.

In my words: if you need such system, it’s better to have recording drives distributed on each recording machine, have RAID 5 there, and additionally have NAS (or some other form of storage) to DUPLICATE everything.

Bandwidth issues

At a configuration with 4 channels recorded at one machine, and with above mentioned data rate for video and audio, each machine will produce 5 megabit of content every second. Roughly, that is .5 megabyte. Even ZIP drive could almost handle that. However, if you have 10 times that (for 10 recorders) and have central storage for the whole bunch of channels, that is 5 megabytes of data at a constant rate that never stops.

Consider that central storage in question is NAS has hard drives configured in RAID 5. That means that it will have to receive, calculate parity for, move the drive heads, write to drive, … It will be very busy NAS, and with everything else in mind, it won’t have a second of a break. Add to that occasional reading of the content from the archive diggers, and you’ll soon figure out that the NAS will have to take it all itself.

On the other hand, archive access applications such as VideoPhill Player doesn’t have anything against having the channels on different recorder machines.

Conclusion (for bandwidth issues) – having each recorder machine handle both encoding and storage for 4 channels will reduce single point of stress for both recording and the archive access.

Overall…

Having dumped my intuition in this few paragraphs, I hope that I presented case that is strong enough against having NAS for video logger/archive storage. Nike Air Max Goedkoop Again, everything said is from my experience on the subject, and I’m no storage expert who will talk petabytes, just a simple consultant trying to get my clients best bang for the buck.

Tags: , , , ,
Posted in tech | 1 Comment »