TerminalCalm commited on
Commit
09b83c3
·
verified ·
1 Parent(s): 4aaface

feat: finished readme

Browse files
Files changed (3) hide show
  1. README.md +16 -1
  2. app.py +15 -11
  3. src/mcp/video_tools.py +46 -19
README.md CHANGED
@@ -7,17 +7,32 @@ sdk: gradio
7
  app_file: app.py
8
  pinned: false
9
  hf_oauth: true
 
10
  ---
11
 
12
  # June 2025 MCP Project
13
 
 
 
14
  This is a Gradio application demonstrating Model-Context-Protocol (MCP) with video processing tools, powered by local Ollama models or a remote Hugging Face model.
15
 
 
 
 
 
 
 
 
 
16
  ## Features
17
 
18
  - **FFmpeg Check:** Verifies that FFmpeg is installed.
19
  - **Video Uploader:** Upload and validate MP4 files.
20
  - **Manual Tools:** Extract the first and last frames of a video.
21
  - **LLM Integration:** Connect to Ollama or Hugging Face.
22
- - **Tool-Calling:** Use natural language to command the LLM to execute video tools.
23
  - **Hugging Face OAuth:** Users can log in with their own HF accounts to use the remote LLM.
 
 
 
 
 
7
  app_file: app.py
8
  pinned: false
9
  hf_oauth: true
10
+ tags: [mcp-server-track, agent-demo-track, video-processing]
11
  ---
12
 
13
  # June 2025 MCP Project
14
 
15
+ Features:
16
+
17
  This is a Gradio application demonstrating Model-Context-Protocol (MCP) with video processing tools, powered by local Ollama models or a remote Hugging Face model.
18
 
19
+ For a little context here: this is a very simple project, built for the purpose of learning about Gradio a little bit more, since I'm more used to working directly with React, React-Native and even ComfyUI on the frontend. Also I wanted to test out MCP, get used to Hugging Face's API, play around with LLM tool calling, and more. And on top of it all, there's some vibe coding going on.
20
+
21
+ All in all, it's been a great experience, even though this is a very basic project compared to what else is going to be submitted during this Hackathon.
22
+
23
+ One last thing: this was built primarily for running locally, rather than on HF itself, since I don't want to tie anything to my HF API key without fully understanding the implications of that. But this is a straightforward project that should be easy to run locally, if anyone's inclined to go that far with it.
24
+
25
+ Thanks for reading all this, I really didn't expect anyone to, ha ha.
26
+
27
  ## Features
28
 
29
  - **FFmpeg Check:** Verifies that FFmpeg is installed.
30
  - **Video Uploader:** Upload and validate MP4 files.
31
  - **Manual Tools:** Extract the first and last frames of a video.
32
  - **LLM Integration:** Connect to Ollama or Hugging Face.
33
+ - **Tool-Calling:** Use natural language to command the LLM to execute video tools, specifically the first frame, last frame, and convert to gif with max resolution and FPS settings (50fps max, 100px min, resolution is in pixel format and chooses the max value of width or height, whatever is greater, to reduce/increase the resolution to, scaling proportionally)
34
  - **Hugging Face OAuth:** Users can log in with their own HF accounts to use the remote LLM.
35
+
36
+ ## Usage Notes
37
+
38
+ - Run the repo locally. Drag in an mp4. Set up your Ollama configuation on the LLM configuration tab, or login to Hugging Face locall. Ideally in either case you're going to want to target something like llama3.2:3b-instruct to get this going, which is what I've developed on. Preferred model and configuration will be sagved locally. From there, go to the LLM Video Commands tab, type in your prompt ("Get me the last Frame", "Get me the first frame", "Make this mp4 into a gif with a max resolution of 300 and an FPS of 50") and hit the button. It should get you what you asked for.
app.py CHANGED
@@ -71,7 +71,7 @@ with gr.Blocks() as demo:
71
  uploaded_video_path_state = gr.State("")
72
 
73
  with gr.Tabs():
74
- # --- Setup & Video Upload Tab ---
75
  with gr.Tab("Setup & Video"):
76
  gr.Markdown("## System Status")
77
  with gr.Row():
@@ -84,17 +84,21 @@ with gr.Blocks() as demo:
84
  with gr.Column(scale=1):
85
  gr.Markdown("### Upload Video")
86
  file_input = gr.File(label="Upload MP4", file_types=[".mp4"])
87
- video_output = gr.Video(label="Preview", interactive=False)
88
  upload_status_text = gr.Textbox(label="Upload Status", interactive=False)
89
-
90
- with gr.Column(scale=2, visible=False) as video_tools_group:
91
- gr.Markdown("### Manual Frame Extraction")
92
- with gr.Row():
93
- get_first_frame_btn = gr.Button("Get First Frame")
94
- get_last_frame_btn = gr.Button("Get Last Frame")
95
- with gr.Row():
96
- first_frame_img = gr.Image(label="First Frame", type="filepath", interactive=False)
97
- last_frame_img = gr.Image(label="Last Frame", type="filepath", interactive=False)
 
 
 
 
98
 
99
  with gr.Tab("LLM Video Commands"):
100
  gr.Markdown("## Test MCP Tool Calls with an LLM")
 
71
  uploaded_video_path_state = gr.State("")
72
 
73
  with gr.Tabs():
74
+ # --- Setup & Video Tab ---
75
  with gr.Tab("Setup & Video"):
76
  gr.Markdown("## System Status")
77
  with gr.Row():
 
84
  with gr.Column(scale=1):
85
  gr.Markdown("### Upload Video")
86
  file_input = gr.File(label="Upload MP4", file_types=[".mp4"])
87
+ video_output = gr.Video(label="Preview", interactive=False, height="50vh")
88
  upload_status_text = gr.Textbox(label="Upload Status", interactive=False)
89
+ with gr.Column(scale=1):
90
+ # This empty column will take up the other 50% of the space
91
+ pass
92
+
93
+ with gr.Tab("Debug"):
94
+ with gr.Column(scale=2, visible=False) as video_tools_group:
95
+ gr.Markdown("### Manual Frame Extraction")
96
+ with gr.Row():
97
+ get_first_frame_btn = gr.Button("Get First Frame")
98
+ get_last_frame_btn = gr.Button("Get Last Frame")
99
+ with gr.Row():
100
+ first_frame_img = gr.Image(label="First Frame", type="filepath", interactive=False)
101
+ last_frame_img = gr.Image(label="Last Frame", type="filepath", interactive=False)
102
 
103
  with gr.Tab("LLM Video Commands"):
104
  gr.Markdown("## Test MCP Tool Calls with an LLM")
src/mcp/video_tools.py CHANGED
@@ -51,7 +51,8 @@ def getFirstFrame(video_path: str) -> str:
51
 
52
  def getLastFrame(video_path: str) -> str:
53
  """
54
- Extracts the last frame from an MP4 video file using FFmpeg.
 
55
 
56
  Args:
57
  video_path: The relative path to the MP4 file (e.g., 'tmp/my_video.mp4').
@@ -72,32 +73,58 @@ def getLastFrame(video_path: str) -> str:
72
  file_name_without_ext = os.path.splitext(base_name)[0]
73
  output_frame_path = os.path.join(output_dir, f"{file_name_without_ext}_last_frame.jpg")
74
 
75
- # Construct and run the ffmpeg command
76
- # -sseof -1 seeks to 1 second before the end of the file to grab the last frame.
77
- command = [
78
- "ffmpeg",
79
- "-sseof", "-1", # Seek to 1s before the end.
80
- "-i", video_path, # Input file
81
- "-vframes", "1", # Extract only one frame
82
- "-q:v", "2", # Output quality (2 is high)
83
- "-y", # Overwrite output file if it exists
84
- output_frame_path
85
- ]
86
-
87
  try:
88
- # Use subprocess.run to execute the command
89
- result = subprocess.run(
90
- command,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
91
  capture_output=True,
92
  text=True,
93
  check=True
94
  )
 
95
  return output_frame_path
 
96
  except FileNotFoundError:
97
- return "Error: ffmpeg is not installed or not found in the system's PATH."
98
  except subprocess.CalledProcessError as e:
99
- # Provide the stderr from ffmpeg for easier debugging
100
- return f"Error: Could not extract last frame. {e}"
 
 
 
101
 
102
  def convert_mp4_to_gif(video_path: str, maxResolution: int = 500, fps: int = 15, pingpong: bool = False) -> str:
103
  """
 
51
 
52
  def getLastFrame(video_path: str) -> str:
53
  """
54
+ Extracts the very last frame from a video file using a precise method
55
+ with ffprobe and ffmpeg.
56
 
57
  Args:
58
  video_path: The relative path to the MP4 file (e.g., 'tmp/my_video.mp4').
 
73
  file_name_without_ext = os.path.splitext(base_name)[0]
74
  output_frame_path = os.path.join(output_dir, f"{file_name_without_ext}_last_frame.jpg")
75
 
 
 
 
 
 
 
 
 
 
 
 
 
76
  try:
77
+ # Step 1: Use ffprobe to get the exact number of frames
78
+ ffprobe_command = [
79
+ "ffprobe",
80
+ "-v", "error",
81
+ "-select_streams", "v:0",
82
+ "-show_entries", "stream=nb_frames",
83
+ "-of", "default=nokey=1:noprint_wrappers=1",
84
+ video_path
85
+ ]
86
+
87
+ probe_result = subprocess.run(
88
+ ffprobe_command,
89
+ capture_output=True,
90
+ text=True,
91
+ check=True
92
+ )
93
+
94
+ total_frames = int(probe_result.stdout.strip())
95
+ if total_frames <= 0:
96
+ return "Error: Video contains no frames."
97
+
98
+ last_frame_index = total_frames - 1
99
+
100
+ # Step 2: Use ffmpeg to extract the last frame by its index
101
+ ffmpeg_command = [
102
+ "ffmpeg",
103
+ "-i", video_path,
104
+ "-vf", f"select='eq(n,{last_frame_index})'",
105
+ "-vframes", "1",
106
+ "-q:v", "2",
107
+ "-y",
108
+ output_frame_path
109
+ ]
110
+
111
+ subprocess.run(
112
+ ffmpeg_command,
113
  capture_output=True,
114
  text=True,
115
  check=True
116
  )
117
+
118
  return output_frame_path
119
+
120
  except FileNotFoundError:
121
+ return "Error: ffmpeg or ffprobe is not installed or not found in the system's PATH."
122
  except subprocess.CalledProcessError as e:
123
+ # Provide stderr for better debugging
124
+ error_details = e.stderr.strip() if e.stderr else "No stderr output."
125
+ return f"Error during frame extraction: {error_details}"
126
+ except (ValueError, TypeError):
127
+ return "Error: Could not parse the number of frames from ffprobe output."
128
 
129
  def convert_mp4_to_gif(video_path: str, maxResolution: int = 500, fps: int = 15, pingpong: bool = False) -> str:
130
  """